@MikeElgan Hmm I wonder who's benefiting from all this wasteful bullshit
C
crystal@hachyderm.io
@crystal@hachyderm.io
Posts
-
Now you can keep track of how many billions the AI companies are losing on AI. -
Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because@inthehands I think if you really want to sell that point, you should explicitly disallow googlebot in robots.txt, then also setup the middleware to respond with 404 to any other URI requested by googlebot