<![CDATA[The bright #LLM future, next part.]]>The bright future, next part.

git.gentoo.org is now effectively dead, being DDoS-ed by almost a million different IPs every day. Most of them are just performing a single request at a totally random URL. How are people supposed to deal with that? How can we distinguish a legitimate user who hit some URL from a scraper that distributes its operations over thousands of IP addresses?

If you use LLM crap, you're part of the problem. You support these bastards. You should be ashamed of yourself.

]]>https://board.circlewithadot.net/topic/90b58370-d13b-4069-832e-34354ed2f562/the-bright-llm-future-next-part.RSS for NodeFri, 01 May 2026 07:58:57 GMTSat, 25 Apr 2026 12:15:33 GMT60<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 19:45:09 GMT]]>@phf @mgorny yes.

It’s actually one of the more damnable bashisms because…

foo &>bar

gets parsed as

foo &
>bar

that is, send foo into the background and truncate bar; it is always better to just transform this bashism into a standard redirection, even if you know you have GNU bash:

foo >bar 2>&1

]]>https://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ5N621SRESG02SHGM6VC63Khttps://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ5N621SRESG02SHGM6VC63KSun, 26 Apr 2026 19:45:09 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 19:37:24 GMT]]>@mirabilos I think I get it. It's a bash-ism to redirect stdout and stderr and in (da)sh that doesn't work? @mgorny

]]>https://board.circlewithadot.net/post/https://dmv.community/users/phf/statuses/116472692347419135https://board.circlewithadot.net/post/https://dmv.community/users/phf/statuses/116472692347419135Sun, 26 Apr 2026 19:37:24 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 19:35:00 GMT]]>@mgorny it does not help pointing to people using LLMs for legitimate reasons. It's other people using those same tools but then for nefarious purposes.
I use user-agent filtering and put Anubis in front of the Slackware git infrastructure, and that has helped immensely.
I eventually got git.gentoo.org to render and gosh! That's a lot of repositories there. Would it be an idea to distribute the cgit interface over multiple front-end servers? Like, moving all user repos to a different server?

]]>https://board.circlewithadot.net/post/https://fosstodon.org/users/alien/statuses/116472682934122650https://board.circlewithadot.net/post/https://fosstodon.org/users/alien/statuses/116472682934122650Sun, 26 Apr 2026 19:35:00 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 19:26:55 GMT]]>@algernon @mgorny cc @alderwick re the current bot attacks on our forge

"https://chronicles.mad-scientist.club/tales/surviving-the-crawlers/#three-ifs-in-a-trenchcoat"

]]>https://board.circlewithadot.net/post/https://merveilles.town/users/cblgh/statuses/116472651150590826https://board.circlewithadot.net/post/https://merveilles.town/users/cblgh/statuses/116472651150590826Sun, 26 Apr 2026 19:26:55 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 19:21:57 GMT]]>@mgorny @phf yes, the eedirection. Can explain more layer if needed, from the laptop.t

]]>https://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ5KVJBP03263J4M0PHSAP72https://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ5KVJBP03263J4M0PHSAP72Sun, 26 Apr 2026 19:21:57 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 19:20:50 GMT]]>@js @mgorny it’s also slop.

(And it’s not been stopping LLMs for months now.)

]]>https://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ5KSHHK2WMQX9YMKDVPTD4Hhttps://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ5KSHHK2WMQX9YMKDVPTD4HSun, 26 Apr 2026 19:20:50 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 18:10:05 GMT]]>@mirabilos It does not? Sure deleted a lot of files when I tried it in a container... 😬 Please to "edumacate" me? Or do you refer to the redirection? @mgorny

]]>https://board.circlewithadot.net/post/https://dmv.community/users/phf/statuses/116472349018258853https://board.circlewithadot.net/post/https://dmv.community/users/phf/statuses/116472349018258853Sun, 26 Apr 2026 18:10:05 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 18:03:49 GMT]]>@mirabilos @mgorny Wat? It’s stopping LLMs.]]>https://board.circlewithadot.net/post/https://ap.nil.im/objects/b02bf142-e615-4066-8f87-2e8ec9b5a1bfhttps://board.circlewithadot.net/post/https://ap.nil.im/objects/b02bf142-e615-4066-8f87-2e8ec9b5a1bfSun, 26 Apr 2026 18:03:49 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 17:34:05 GMT]]>@davidgerard thank you!

]]>https://board.circlewithadot.net/post/https://ciberlandia.pt/users/villares/statuses/116472207468582858https://board.circlewithadot.net/post/https://ciberlandia.pt/users/villares/statuses/116472207468582858Sun, 26 Apr 2026 17:34:05 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 17:17:46 GMT]]>@villares no, but I went to https://iocaine.madhouse-project.org/ and faffed about a bit. I used iocaine 3 out of the box. i use nginx so i had to figure out the correct config. i added exceptions for some specific user-agents I wanted to let through.

]]>https://board.circlewithadot.net/post/https://circumstances.run/users/davidgerard/statuses/116472143336678766https://board.circlewithadot.net/post/https://circumstances.run/users/davidgerard/statuses/116472143336678766Sun, 26 Apr 2026 17:17:46 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 16:45:43 GMT]]>@davidgerard would you have pointers to "how to guides" for less savvy people? I have a shared hosting account on a web hosting service, I feel like I need to protect myself from these bots and I'm totally lost.

]]>https://board.circlewithadot.net/post/https://ciberlandia.pt/users/villares/statuses/116472017310019236https://board.circlewithadot.net/post/https://ciberlandia.pt/users/villares/statuses/116472017310019236Sun, 26 Apr 2026 16:45:43 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 16:03:57 GMT]]>@Kyebr Yes, and that's fine: tools that don't try to pretend  to be browsers get through the Three Ifs (and iocaine) just fine.

@mgorny

]]>https://board.circlewithadot.net/post/https://come-from.mad-scientist.club/users/algernon/statuses/01KQ58H0QJ5NH8118109742M55https://board.circlewithadot.net/post/https://come-from.mad-scientist.club/users/algernon/statuses/01KQ58H0QJ5NH8118109742M55Sun, 26 Apr 2026 16:03:57 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 16:02:06 GMT]]>@algernon @mgorny I’m guessing that most of the traffic for git.gentoo.org would be by tools on behalf of users. And not through browsers.

Anyway, great article. I’m thinking about implementing the same on my site.

]]>https://board.circlewithadot.net/post/https://hachyderm.io/users/Kyebr/statuses/116471845807705690https://board.circlewithadot.net/post/https://hachyderm.io/users/Kyebr/statuses/116471845807705690Sun, 26 Apr 2026 16:02:06 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 15:59:31 GMT]]>@mgorny @phf that line does not do what you think it does, in sh…

]]>https://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ588XF1X337RAS9ME0CR50Ehttps://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ588XF1X337RAS9ME0CR50ESun, 26 Apr 2026 15:59:31 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 15:58:45 GMT]]>@mgorny @js Anubis is LLM slop…

]]>https://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ587G25H8WXMJX7G9MDFQ7Hhttps://board.circlewithadot.net/post/https://toot.mirbsd.org/users/mirabilos/statuses/01KQ587G25H8WXMJX7G9MDFQ7HSun, 26 Apr 2026 15:58:45 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 14:11:01 GMT]]>@algernon 👍 @mgorny

]]>https://board.circlewithadot.net/post/https://mastodon.online/users/NatureMC/statuses/116471409013087680https://board.circlewithadot.net/post/https://mastodon.online/users/NatureMC/statuses/116471409013087680Sun, 26 Apr 2026 14:11:01 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 13:27:32 GMT]]>@saxnot @mgorny 100 GB billion laughs attack compressed to 80 KB.]]>https://board.circlewithadot.net/post/https://ap.nil.im/objects/ee63f97a-2e78-49ee-999d-84c6d0bb969ahttps://board.circlewithadot.net/post/https://ap.nil.im/objects/ee63f97a-2e78-49ee-999d-84c6d0bb969aSun, 26 Apr 2026 13:27:32 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 13:26:55 GMT]]>@mgorny I guess then you can be glad your cat only crashed your browser and didn’t delete your home.]]>https://board.circlewithadot.net/post/https://ap.nil.im/objects/9d176aa2-54d0-4fe7-9076-06ee05e6e25bhttps://board.circlewithadot.net/post/https://ap.nil.im/objects/9d176aa2-54d0-4fe7-9076-06ee05e6e25bSun, 26 Apr 2026 13:26:55 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 12:25:20 GMT]]>@kigelia, yep. We're hosting a few huge repos (and a lot of small ones), so the load caused by crawling everything randomly (including stuff such as commit histories filtered by individual files, git blames and other stuff that's entirely redundant) prevents real people from using the service.

]]>https://board.circlewithadot.net/post/https://social.treehouse.systems/users/mgorny/statuses/116470993452293003https://board.circlewithadot.net/post/https://social.treehouse.systems/users/mgorny/statuses/116470993452293003Sun, 26 Apr 2026 12:25:20 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 12:21:11 GMT]]>@ericalaeta @mgorny

Everyone will have to introduce APIs that can accept much more traffic.

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/machocam/statuses/116470977068402685https://board.circlewithadot.net/post/https://mastodon.social/users/machocam/statuses/116470977068402685Sun, 26 Apr 2026 12:21:11 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 12:12:45 GMT]]>@js @mgorny I would definitely hit that, just to see what happens 😬 definitely would also curl afterwards

]]>https://board.circlewithadot.net/post/https://lviv.social/users/bgnfu7re/statuses/116470943934478598https://board.circlewithadot.net/post/https://lviv.social/users/bgnfu7re/statuses/116470943934478598Sun, 26 Apr 2026 12:12:45 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 12:09:51 GMT]]>@mgorny yeah we have the same problem with our hosting product at @ubernauten. It's sucks to spend so much time working to mitigate the bad behavior of others. Sadly we have no solution either.

]]>https://board.circlewithadot.net/post/https://chaos.social/users/leah/statuses/116470932508462388https://board.circlewithadot.net/post/https://chaos.social/users/leah/statuses/116470932508462388Sun, 26 Apr 2026 12:09:51 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 11:49:19 GMT]]>@mgorny I blocked it all based on user agent. They use a useragent which is quite rare. So I blocked that.

]]>https://board.circlewithadot.net/post/https://mastodon.opencloud.lu/users/gunstick/statuses/116470851806093884https://board.circlewithadot.net/post/https://mastodon.opencloud.lu/users/gunstick/statuses/116470851806093884Sun, 26 Apr 2026 11:49:19 GMT<![CDATA[Reply to The bright #LLM future, next part. on Sun, 26 Apr 2026 11:45:21 GMT]]>@kigelia @mgorny that is certainly a problem that is occurring, yes. Since code generation is a primary use case, public code stores are an example of sites that are getting hit badly, which in turn is making open source less viable (alongside junk "fixes" and other noise)

]]>https://board.circlewithadot.net/post/https://merveilles.town/users/swift/statuses/116470836189805143https://board.circlewithadot.net/post/https://merveilles.town/users/swift/statuses/116470836189805143Sun, 26 Apr 2026 11:45:21 GMT