Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because

[dotstdy@mastodon.social]

@joe @inthehands @ShadSterling i like to prop the demon core open with a screwdriver and point the aperture at the internet

[maddiem4@raphus.social]

@inthehands All this, and a little more! When Google *does* start ignoring robots.txt and other mechanisms, that's another victory for us, not them, even if it means we have to react to it.

Not all of Google's infrastructure is servers in a giant building, or software systems running on top of it, or even offices full of stressed out tech workers. Part of their infrastructure, the cladding on the castle walls, is their false pretense of being good citizens on the internet. When we call their bluff and they eventually drop the pretense, that's us getting them to tear down the outer layers of the castle themselves. We know what they are, and we can make them admit it, and that's power.

[mathaetaes@infosec.exchange]

@inthehands I know of at least one professional artist who has deliberately poisoned their images, in an attempt to deter AI scraping (mostly because the scrapers blast her small site and effectively DoS it). If they follow robots.txt, they're not affected... but they were already ignoring robots.txt

I just read an IARPA paper that said poisoning as little as .1% of training data can disrupt a model. If content creators choose to deliberately poison content that they ask not to be scraped, it might be a nice way to deter bad behavior.

The tools I know of work on imagery, but with effort people may come up with stuff that works on data as well. E.g., burying base64-encoded malicious prompts in your text, posting tables as poisoned images rather than text, etc.

Seems like we should start organizing and taking firm action now, before AI companies start buying politicians and making such defenses illegal.

[sennoma@chaos.social]

@dotstdy You saw what happened to the last guy who did that, right?

*blue flash*

@joe @inthehands @ShadSterling

[sennoma@chaos.social]

@inthehands @adamshostack

What they'll *need* is your wallet.

[schamschula@mastodon.social]

@inthehands If they ignore robots.txt, they will be added to the block list in nginx.conf. My robots.txt has a note stating as much. There is plenty of company there!

[dotstdy@mastodon.social]

@sennoma @joe @inthehands @ShadSterling it's okay i put an `unsafe` label on the door

[crystal@hachyderm.io]

@inthehands I think if you really want to sell that point, you should explicitly disallow googlebot in robots.txt, then also setup the middleware to respond with 404 to any other URI requested by googlebot

[khm@hj.9fs.net]

I return 402 Payment Required to googlebot user agents

[skellysoft@mastodon.gamedev.place]

@inthehands hard agree (even though I'm thinking of this in broader terms). It can be so difficult to keep the will to do so, in practice... but it's important to do it whenever we can.

[hyc@mastodon.social]

@khm @inthehands Oooh. I like it. Gonna have to add that to my nginx configs instead of 403. And add a Monero wallet address in the text content

[ronanmcd@mastodon.green]

@inthehands this is a great point

[macronaut@mas.to]

@hyc @khm @inthehands is there a “how to” on this that one can use to update their web server/site?

[114@tech.lgbt]

@ShadowJonathan @inthehands agree. I don't think defense is the best reaction to sustain a healthy internet. this rhetoric has been untrue since... Google (other similar corps).

random offensive approach such as collective data poisoning, public exposé, factual based journalism, education, jailtime, guillotine & other accountability and positive encouragement should coexist to foster the internet to recover better

[jef@mastodon.social]

@inthehands How about blocking port 80 & 443 access from Google's netblocks? 142.250.0.0/15 for starters.

[hyc@mastodon.social]

@macronaut @khm @inthehands Currently I've created in nginx/conf/server_extra/block-useragent.conf:

if ($http_user_agent ~* meta-externalagent) {
return 403;
}

And I've added an
include server_extra/*.conf;

in my site's server{} config.

[hyc@mastodon.social]

@macronaut @khm @inthehands the update would change that 403 to a 402. And add "error_page 402 /402.html;" to the server{} config, and create the /402.html file in the docroot containing whatever desired message.

[darwinwoodka@mastodon.social]

@inthehands

Damn straight.

Keep the faith, baby. And take care of each other.

[marco_m_aus_f@freiburg.social]

@inthehands Dissallow in robots.txt and install iocaine or anubis or another AI-poisoning software, should they ignore the robots.txt.

[donaldball@triangletoot.party]

@mjd @cceckman No. When you publish anything without a specific declaration otherwise, it is automatically covered by copyright protections. Those protections do *not* allow the extensive (ab)uses you cite, only those permitted by the fair use doctrine.