Still, the blog post explicitly mentions "In addition to fixing the 271 bugs identified by Claude Mythos Preview in the 150 release, we’ve shipped more of these fixes in 149.0.2, 150.0.1, and 150.0.2.", so >270 attributed to #Mythos *alone*.

llm tools found security issues in curl? doubt

I was just pointing out this constant theme. The only thing that ever is made public is the fully-polished, human-vetted final result. They carefully hide all other details and the press don’t care.

@bagder

There were lots of shenanigans afaik.

• No one who has worked with raw Mythos output has ever written about it.
• No one who has written about it has ever used it.

They would much rather have @bagder writing about it because his opinion carries weight. That means he can’t have direct access. To give him access, they’d demand to gag him with an NDA, like everyone else who has access.

This technique of making readers mentally fill in the gaps between what is verifiable and what is claimed is genius marketing and really dishonest. But we have come to expect systematic and casual dishonesty from these companies.

• 5M lines of rust
• 9M lines of C and C++
• 200k lines of assembly
• 2M lines of python

so like, without looking at anything else, firefox is significantly bigger

curl is the ship of Theseus not once, not twice, but four times

1. Put back all the curl security issues previously found by LLM tools by dropping the fix commits from history or otherwise obfuscating the revert.

2. Feed the re-vulnerabilized repo to a selection of models and see what are the cheapest ones (by memory, time and/or monetary cost) that can find, say, 50%/75%/100% of the issues found by the warehouse-scale "foundation models".

Feels like a large part of the current results should be doable with significantly smaller resources, because being trained on every tweet and reddit post and libgen book ever is not obviously related to the task.

Especially if it's subscription-based, as these models seem to be good at finding only specific sets of problems and then dry out, but even 10k per use is really gov or big corpo territory.

@kleisli @bagder