CIRCLE WITH A DOT

@freddy @gabrielesvelto Also, it looks to me that fuzzing requires more human setup of what part of code to fuzz and how to deal with stuff like checksums whereas reportedly LLMs can deal with less specific harnesses and figure out how to fill in checksums.

Or folks go LOL at security incidents or code quality at an LLM company. Irrelevant to whether their model can find security bugs. The way this works is that you have a non-LLM oracle like ASAN. If the model found a way to trigger the oracle, then it’s not really productive to argue that it didn’t.

Why even post this considering the predictable hate? Because denial about the situation does not make users safer from attacks.

Then there’s the dismissal that, yes, LLMs now find security bugs, but the bugs could have been found by other methods. But evidently defenders hadn’t actually found them by other methods. (Unknown what attackers had already found.)

Or folks find it objectionable that the new capability has been made available to attackers and the proposed cure is to pay for access to the same LLM. But that does make the existence of the capability untrue.

And, yes, the Anthropic Mythos post fits a previously-seen pattern of “AI” companies marketing by danger, but saying that it’s marketing does not refute what the models that are already generally offered can do.

And people act like their own conjecture is more informative than what people from multiple projects that deal with security bug reports say. See e.g. https://mastodon.social/@bagder/116363034479757682 .

I’m seeing a lot of denial and logical fallacies on Mastodon about LLM capability to find security bugs.

I get it that when folks have concluded that LLMs are harmful, they want to believe that LLMs fail at everything. But a list of correctly-identified bad things about LLMs does not logically imply that LLMs can’t find security bugs.

@mkj Nope, due to the British Standard Time Act of 1968. Thanks to your toot, I realized that my fix was wrong!

(This looks like time zone satire or April Fools, but, unfortunately, I’m not joking.)

The time zone and daylight-saving time fans out there will probably appreciate the user impact description that I wrote for release management:

“Users whose birthday is before the end of October 1971 on a day when the UK was not observing daylight-saving time can't sign up for online banking at a particular British bank.”

It turned out that some time zones that use “GMT” as their abbreviation in the same style as CET for Central European Time or PDT for Pacific Daylight-saving Time previously got “GMT” by a different mechanism, so an intentional change to that mechanism had unintended effects.

Today in Web compat: Firefox and Safari are ahead of Chrome in ICU4C version and upsteam ICU4C changed the formatting of zero offset from GMT. This broke birthday date validation for a UK based site for birthdays before 1970 in Firefox and, on 26.x Apple OSs, in Safari, because the site performs a formatting-based check on the time zone of London on the date to be validated and the UK has changed time zone rules along the way.