How to block "Uknown robots identified by any ^&^%$#% existent"?

Support
#1

My website is plagued by two robots that have no name, no IP nothing at all.
The first one is the well known

Unknown robot identified by bot slash*

and the other one is the

Unknown robot (identified by hit on robots.txt)

I used to block the first one that is the most persistent on my .htaccess file but that stopped working at some point.

I don’t want also to add yet another plugin just for blocking these damn bots. My website seems to be just fine as it is now. It has the golden ratio of plugins.

Looking up and down online I found this discussion.

Is it of any valid, I mean can I block these bots from index.php that is modified for CP or any attempt to edit it, it will bring the universe down??

1 Like
#2

This is the best (most efficient) way to block small numbers of bots.

You shouldn’t put this code in index.php because it will be overwritten the next time ClassicPress is updated.

You can put it in e.g. wp-content/mu-plugins/block-bad-bots.php, and it looks like the code you linked should work fine there. Don’t forget the opening <?php tag on the first line.

All bots (all visitors to your site in general) do have an IP address. They should also be sending a User-Agent header which is one way that bot detection works. You can find the raw values for both of these pieces of information in the server logs provided by your web host.

3 Likes
#3

Thank you. I’ll try it to see if it works.

Well… I don’t want to see this damn bot with the slash* ever again on my website. I hate it. It is the ultimate bandwidth eater.

And it is out there for countless years.
Who is running this bot btw?
Where it comes from and what is its purpose?

#4

If you’re able to, try using modsec. It’s what I use and has cut down on the number of bad bots significantly (and I include the likes of ahrefs in that list). It may be that it’s something you need to discuss with your hosting company.

modsec will also provide an IP address for each bad bot.

But be warned, if it’s not something you’ve used before, it can be a bit difficult (and somewhat dangerous) to set up.

#6

Thank you for the suggestion, but I will not try it. If something goes wrong I will not be able to fix it and I can’t even think to spend time fixing my website. I already spend a considerable amount of time and effort to update it, time that I could use it to paint.

This website is there to show my art. And my aim is to improve my art not the website.
I think that it is more than good the way it is if you think that I made it from scratch and by reading online tutorials.

I’m very proud for the final result but I don’t want to start messing up with it. It works. Let it be. LOL

3 Likes
#7

Please DON’T close this thread because I haven’t manage to figure out how to do it yet. :thinking:

#8

Have you tried all the suggested fixes? If you’re unable to try them, you may be stuck with the bot traffic.

#9

I don’t know what modsec is… lol

Wait to RTFM please!

#10

Emphasis added.

1 Like
#11

I got this. That is the reason why I asked to leave the thread as it is and I’ll be back.

1 Like
#12

@Marialena.S Found an interesting plugin at Blackhole for Bad Bots that sets a trap for bots. It adds an entry in the robots.txt file and if a Bot follows it, you can configure what happens.

I set it to display a message and send me an email. Using a VPN, I went to the trap page and got greeted with the red message. Refreshing the page then displays my custom message. The most important Bots are already whitelisted by the plugin and do not get blocked.

blackhole
blackhole761×881 78.7 KB

blackhole2
blackhole2953×120 10.1 KB

Turned off the VPN and simply went to the plugin and removed myself. Will be interesting to see how well it works.

1 Like
#13

I often read Jeff Star’s blog at Perishable Press. Some of the “tricks” he suggests are quite useful and amusing. Here’s a short article that gives a bit of an insight to the Blackhole plugin.

3 Likes
#14

@Aussie I think that I have used this plugin at some point in the past. The thing that I don’t understand is why that plugin leaves a message for the bot. Can the bot read it perhaps?? lol
@1stepforward Thank you … I’ll check it out and I’ll be back. :slight_smile:

#15

Bots are operated by humans and designed to find information on your site. Bad Bots hope to find sensitive data such as credit card numbers.

So to answer your question, yes they will report back the message to the owner.

The message is a courtesy in case the Bot is actually well intentioned. You can choose to simply send them straight to the blackhole if you want.

#16

I think that it is not very wise to inform the operator of amalicious bot that you have set up a trap and that you use a security plugin.

#17

In practice it makes very little difference. This type of “security through obscurity” has little to no benefit, and even some drawbacks that are worth considering. (It is a similar story with things like hiding your version number - many common practices related to web security are unhelpful or wrong.)

  • Some of the users that will see your error message are actually real users. These techniques always have false positives so it is pretty unfriendly to have the site just show a blank page or a generic error. This is also the reason why blocking entire countries or IP ranges is usually a bad idea - you will block legitimate users too.
  • Many bots are just misbehaved rather than malicious, in which case if anyone does happen to see the error message and act on it, then the most likely action is they will fix their bot.
  • For the bots that are malicious (harvesting for email addresses and other spammable info), most of them are “dumb” in the sense that they just look for patterns and don’t bother reporting anything else back to the owner.
  • For the few bots that are malicious and smart about what they are doing, there are hundreds of available detection methods and there is basically no way to hide the fact that you are using a security plugin, or even which one you are using. At that point you probably have a determined and resourceful attacker that is looking at your site specifically, and you are playing a completely different game.

tl;dr the default behavior of showing an error message is probably the best choice.

3 Likes
#18

I think the trick is to use multiple systems. I consider a plugin to be the last line of defence and hardening the server to be the most important consideration. Yes, it may slow down your website very slightly but not enough to make any noticeable difference. Not in my experience anyway.

3 Likes
#19

I see… :thinking:

1 Like
#20

Regarding the protection of my ClassicPress sites (but this is valid for non-CP sites as well), I have recently moved all my sites to CleanTalk. And I am quite pleased with it. Moreover, I had recently issues of odd spam getting through, and their support proved to be very committed for adjusting to very specific needs with a third-party contact form. They seem to be a great team.
I do not know if you have already tried them, but I would consider that, as least for testing. What is great with their spam firewall is that: “Spambots are blocked before they get access to the website, it prevents the loading of pages of a website spam bots, so your web server doesn’t need to run all the scripts on these pages. This can reduce the load on the database and web server.”
On two of my ClassicPress sites, I have Shield installed along with CleanTalk, and I have not come across issues until now.

1 Like