@inthehands One of the things I've done recently is to bring enforcing robots.txt within my webserver engine. The /robots.txt itself still exists; the vast majority of it is a list of bots that are `Disallow: /` .
I still get a few of these bots attempting to hit the site, so it's definitely doing something.