@RnDanger @inthehands well, we don’t know and we will see. My guess are separate scrapers (officially) and a lot of mistrust (are there others?) and masses of unidentified scrapers. Nevertheless, Google can better afford to play by the rules, since hey already own the largest index. Think also of Video etc. Will volume win the war? Or quality and freshness? Etc. Future is difficult.
K
korrupt@nrw.social
@korrupt@nrw.social
Posts
-
Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because -
Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because@inthehands meta noindex it is, definitely. robots disallow can actually hurt the process, since google cannot access the file with the noindex header and therefore won't deindex.
btw, they do indeed respect noindex and robots.txt ATM, since its qute easy to check if pages still get found. Then again, you never know what does not show up in search but is used for training (without giving credit, obv.) anyway. As far as i see, google still remains more standard compliant as e.g. OpenAI. -
Level 1: Wasserspender in der Stadt ist kaputt.@blinry das erinnert mich an den ansonsten schwachen fünften Band Anhalter mit dem Raumschiff, in dem das Teil fehlt, das melden soll, wenn ein Teil fehlt.