Interesting read on the way LLM bots retrieve pages from a website

[dendrobatus_azureus@mastodon.bsd.cafe]

Interesting read on the way LLM bots retrieve pages from a website

Explanations are clear precise and surgical

AI traffic vs referral traffic: what nginx logs prove | SurfacedBy

What nginx logs actually prove about AI traffic from ChatGPT, Claude, and Perplexity, plus why Google's AI crawlers leave a different signal than the others.

SurfacedBy (surfacedby.com)

#LLM #AI #slop #nginx #traffic #programming #referral #traffic #networking #robots #txt #claude #chatgpt #bing #meta #metaAI