I want to block a lots of user agents for all websites on my VPS.
Can I add the rules in global Nginx config? If yes where is it?
(btw, I know I can add it to each website, but I need to edit 27+ websites, and in future it can go up)
My rules are:
if ($http_user_agent ~* (2ip\ bot|360Spider|404checker|404enemy|80legs|Abonti|Aboundex|Aboundexbot|Acunetix|ADmantX|adscanner|AdsTxtCrawlerTP|AfD-Verbotsverfahren|AHC|AIBOT|AiHitBot|Aipbot|Alexibot|ALittle\ Client|Alligator|AllSubmitter|AlphaBot|Amazonbot|Anarchie|Anarchy|Anarchy99|Ankit|Anthill|anthropic-ai|Apache-HttpClient|Apexoo|archive.org_bot|arquivo-web-crawler|arquivo.pt|Aspiegel|ASPSeek|Asterias|Atomseobot|Attach|autoemailspider|Awario|AwarioBot|AwarioRssBot|AwarioSmartBot|BackDoorBot|Backlink-Ceck|backlink-check|BacklinkCrawler|BackStreet|BackWeb|Badass|Bandit|Barkrowler|BatchFTP|Battleztar\ Bazinga|BBBike|BDCbot|BDFetch|BetaBot|bidswitchbot|Bigfoot|Bitacle|Black\ Hole|Blackboard|BlackWidow|BLEXBot|Blow|BlowFish|Boardreader|Bolt|BotALot|Brandprotect|Brandwatch|Buck|Buddy|BuiltBotTough|BuiltWith|Bullseye|BunnySlippers|BuzzSumo|Bytedance|Bytespider|cah.io.community|Calculon|CATExplorador|CazoodleBot|CCBot|Cegbfeieh|CensysInspect|CF-UC|ChatGPT-User|check1.exe|CheeseBot|CherryPicker|CheTeam|ChinaClaw|Chlooe|Citoid|Claritybot|clark-crawler|Cliqzbot|Cloud\ mapping|coccocbot|Cocolyzebot|CODE87|Cogentbot|cognitiveseo|cohere-ai|Collector|com.plumanalytics|comscore|Copier|CopyRightCheck|Copyscape|Cosmos|Craftbot|crawl.sogou.com|crawler.feedback|crawler4j|Crawling\ at\ Home\ Project|CrazyWebCrawler|Crescent|CriteoBot|CrunchBot|CSHttp|Curious|Custo|CyotekWebCopy|DatabaseDriverMysqli|DataCha0s|dataforseo.com|DataForSeoBot|dataforseobot|DBLBot|demandbase-bot|Demon|Deusu|Devil|Digincore|DigitalPebble|DIIbot|Dirbuster|dirbuster|Disco|Discobot|Discoverybot|Dispatch|DittoSpyder|DnBCrawler-Analytics|DnyzBot|DomainAppender|DomainCrawler|Domains\ Project|DomainSigmaCrawler|domainsproject.org|DomainStatsBot|DomCopBot|Download\ Wonder|Dragonfly|Drip|DSearch|DTS\ Agent|EasyDL|Ebingbong|eCatch|ECCP/1.0|Ecxi|EirGrabber|EMail\ Siphon|EMail\ Wolf|EroCrawler|evc-batch|Evil|Exabot|Expanse|Express\ WebPictures|ExtLinksBot|Extractor|ExtractorPro|Extreme\ Picture\ Finder|EyeNetIE|Ezooms|FacebookBot|facebookscraper|FDM|FemtosearchBot|FHscan|Fimap|Firefox/7.0|FlashGet|Flunky|Foobot|Freeuploader|FriendlyCrawler|FrontPage|Fuzz|FyberSpider|Fyrebot|G-i-g-a-b-o-t|Gabanzabot|GalaxyBot|Genieo|GermCrawler|Getintent|GetRight|GetWeb|Gigabot|Go-Ahead-Got-It|Go-http-client|Go!Zilla|gopher|Gotit|GoZilla|GPTBot|Grabber|GrabNet|Grafula|GrapeFX|GrapeshotCrawler|GridBot|GT::WWW|Haansoft|HaosouSpider|harvest|Harvest|Havij|HEADMasterSEO|Heritrix|heritrix|Hloader|HMView|HonoluluBot|HTMLparser|HTTP::Lite|HTTrack|Humanlinks|HybridBot|Iblog|Id-search|IDBot|IDBTE4M|IlseBot|Image\ Fetch|Image\ Sucker|imagesift.com|ImagesiftBot|IndeedBot|Indy\ Library|InfoNaviRobot|InfoTekies|instabid|Intelliseek|InterGET|Internet\ Ninja|InternetSeer|internetVista\ monitor|IonCrawl|ips-agent|Iria|IRLbot|isitwp.com|Iskanie|IstellaBot|iubenda-radar|JamesBOT|Java|Jbrofuzz|JennyBot|JetCar|Jetty|JikeSpider|JOC\ Web\ Spider|Joomla|Jorgee|JustView|Jyxobot|Kenjin\ Spider|Keybot\ Translation-Search-Machine|Keyword\ Density|Kinza|Kozmosbot|Lanshanbot|Larbin|Leap|LeechFTP|LeechGet|LexiBot|Lftp|LibWeb|Libwhisker|LieBaoFast|Lightspeedsystems|Likse|Linkbot|linkdexbot|LinkextractorPro|LinkpadBot|LinkScan|LinksManager|LinkWalker|LinqiaMetadataDownloaderBot|LinqiaRSSBot|LinqiaScrapeBot|Lipperhey|Lipperhey\ Spider|Litemage_walker|Lmspider|LNSpiderguy|Ltx71|lwp-request|lwp-trivial|LWP::Simple|Mag-Net|Magnet|magpie-crawler|Mail.RU_Bot|Majestic-SEO|Majestic\ SEO|Majestic12|MarkMonitor|MarkWatch|Mass\ Downloader|Masscan|Mata\ Hari|MauiBot|Mb2345Browser|MBCrawler|MeanPath\ Bot|Meanpathbot|meanpathbot|Mediatoolkitbot|mediawords|MegaIndex.ru|Metauri|MFC_Tear_Sample|MicroMessenger|Microsoft\ Data\ Access|Microsoft\ URL\ Control|MIDown\ tool|MIIxpc|Minefield|Mister\ PiX|MJ12bot|MJ12Bot|Moblie\ Safari|Mojeek|Mojolicious|MolokaiBot|Morfeus\ Fucking\ Scanner|Mozlila|MQQBrowser|Mr.4x3|MSFrontPage|MSIECrawler|Msrabot|MTRobot|muhstik-scan)) {
return 403;
}
if ($http_user_agent ~* (Musobot|Name\ Intelligence|Nameprotect|Navroad|NearSite|Needle|Nessus|Net\ Vampire|NetAnts|Netcraft|netEstate\ NE\ Crawler|NetLyzer|NetMechanic|NetSpider|Nettrack|Netvibes|NetZIP|newspaper|NextGenSearchBot|Nibbler|NICErsPRO|Niki-bot|Nikto|NimbleCrawler|Nimbostratus|Ninja|Nmap|NPbot|Nuclei|Nutch|Octopus|Offline\ Explorer|Offline\ Navigator|omgili|OnCrawl|Openfind|OpenLinkProfiler|OpenVAS|Openvas|OrangeBot|OrangeSpider|OutclicksBot|OutfoxBot|Page\ Analyzer|page\ scorer|PageAnalyzer|PageGrabber|PageScorer|PageThing.com|Pandalytics|pangolin|Panscient|Papa\ Foto|Pavuk|pcBrowser|PECL::HTTP|PeoplePal|Petalbot|PHPCrawl|Pi-Monster|Picscout|Picsearch|PictureFinder|Piepmatz|Pimonster|Pixray|PleaseCrawl|plumanalytics|Pockey|POE-Component-Client-HTTP|polaris\ version|probe-image-size|Probethenet|ProPowerBot|ProWebWalker|Proximic|Psbot|Pu_iN|Pump|PxBroker|PyCurl|python-requests|Python-urllib|QueryN\ Metasearch|Quick-Crawler|Rainbot|RainBot|RankActive|RankActiveLinkBot|RankFlex|RankingBot|RankingBot2|Rankivabot|RankurBot|Re-re|RealDownload|Reaper|RebelMouse|Recorder|RedesScrapy|ReGet|RepoMonkey|Ripper|ripz|RocketCrawler|RSSingBot|s1z.ru|SalesIntelligent|satoristudio.net|SBIder|scalaj-http|scan.lol|ScanAlert|Scanbot|ScoutJet|Scrapy|Screaming|ScreenerBot|ScrepyBot|Searchestate|SearchmetricsBot|Seekport|SeekportBot|SemanticJuice|SentiBot|SenutoBot|seobility|SeobilityBot|seocompany.store|SEOkicks|SEOkicks-Robot|SEOlyticsCrawler|SEOprofiler|seoscanners|SeoSiteCheckup|seostar|SEOstats|serpstatbot|sexsearcher|SeznamBot|Shodan|Siphon|SISTRIX|Site\ Sucker|Sitebeam|sitechecker.pro|SiteCheckerBotCrawler|SiteExplorer|Siteimprove|SiteLockSpider|siteripz|SiteSnagger|SiteSucker|Sitevigil|SlySearch|SmartDownload|SMTBot|Snake|Snapbot|Snoopy|SocialRankIOBot|Sociscraper|Sogou\ web\ spider|sogouspider|Sosospider|Sottopop|sp_auditbot|SpaceBison|Spammen|SpankBot|Spanner|Spbot|Spider_Bot|Spinn3r|spinner|SputnikBot|spyfu|Sqlmap|Sqlworm|Sqworm|Steeler|Stripper|Sucker|Sucuri|SuperBot|SuperHTTP|Surfbot|SurveyBot|Suzuran|Swiftbot|sysscan|Szukacz|T0PHackTeam|T8Abot|tAkeOut|Teleport|TeleportPro|Telesoft|Telesphoreo|Telesphorep|The\ Intraformant|TheNomad|thesis|Thumbor|TightTwatBot|Timpibot|tiny|TinyTestBot|Titan|Toata|Toweyabot|Tracemyfile|Trendiction|trendiction.com|trendiction.de|Trendictionbot|True_Robot|Turingos|Turnitin|TurnitinBot|TwengaBot|Twice|Typhoeus|ubermetrics-technologies.com|UnisterBot|Upflow|URLy.Warning|URLy\ Warning|V-BOT|Vacuum|Vagabondo|VB\ Project|VCI|VelenPublicWebCrawler|VeriCiteCrawler|VidibleScraper|Virusdie|VoidEYE|Voil|Voltron|voyagerx.com|Wallpapers|Wallpapers/3.0|WallpapersHD|WASALive-Bot|WBSearchBot|Web\ Auto|Web\ Collage|Web\ Enhancer|Web\ Fetch|Web\ Fuck|Web\ Pix|Web\ Sauger|Web\ Sucker|Webalta|WebAuto|WebBandit|WebCollage|WebCopier|WEBDAV|WebEnhancer|WebFetch|WebFuck|webgains-bot|WebGo\ IS|WebImageCollector|WebLeacher|WebmasterWorldForumBot|webmeup-crawler|WebPix|webpros.com|webprosbot|WebReaper|WebSauger|Webshag|Website\ Quester|WebsiteExtractor|WebsiteQuester|Webster|WebStripper|WebSucker|WebWhacker|WebZIP|WeSEE|Whack|Whacker|Whatweb|Who.is\ Bot|Widow|WinHTTrack|WiseGuys\ Robot|WISENutbot|Wonderbot|Woobot|Wotbox|wp_is_mobile|Wprecon|WPScan|WWW-Collector-E|WWW-Mechanize|WWW::Mechanize|WWWOFFLE|x09Mozilla|x22Mozilla|Xaldon_WebSpider|Xaldon\ WebSpider|Xenu|xpymep1.exe|YoudaoBot|Zade|Zauba|zauba.io|Zermelo|Zeus|zgrab|Zitebot|ZmEu|ZoomBot|ZoominfoBot|ZumBot|ZyBorg)) {
return 403;
}
I did find something
https://www.xmodulo.com/block-specific-user-agents-nginx-web-server.html
But is for every site.
I was wondering if it can be added to:
