denverdonate
Level 1
Write the reason you're deleting this FAQ
I have tried the following: .htaccess to try and get them out, used robots.txt and a huge list ban IPs by region, ban seeming active bots that might be scrape content, turned Cpanel supercache, enable heavy duty cache, this would slow them in theory, Cpanel to check no-referrer bots, check AWSTATS and look for bots that declare something that they are not.
People say "ban their IP" ---- this actually is not a good suggestion because....do you think the bot will obey robots.txt if they are bad? And, bots don't use the same IP, a badbot of course will not pay attention. One day I saw in real saw 30+ Bingbots active at once, Bing doesn't love me so much to give me their army of bots over and over, those would be mostly badbots.
Are any of you working with a good strategy to get them away from your site?