I keep seeing stories about LLMs finding vulnerabilities.
-
I keep seeing stories about LLMs finding vulnerabilities. Finding vulnerabilities was never the hard part, the hard part is coordinating the disclosure
It looks like LLMs can find vulnerabilities at an alarming pace. Humans aren't great at this sort of thing, it's hard to wade through huge codebases, but there are people who have a talent for vulnerability hunting.
This sort of reminds me of the early days of fuzzing. I remember fuzzing libraries and just giving up because they found too many things to actually handle. Eventually things got better and fuzzing became a lot harder. This will probably happen here too, but it will take years.
What about this coordinating thing?
When you find a security vulnerability, you don't open a bug and move on. You're expected to handle it differently. Even before you report it, you need at a minimum a good reproducer and explanation of the problem. It's also polite to write a patch. These steps are difficult, maybe LLMs can help, we shall see.
Then you contact a project, every project will have a slightly different way they like to have security vulnerabilities reported. You present your evidence and see what happens. It's very common for some discussion to ensue and patch ideas to evolve. This can take days or even weeks. Per vulnerability.
So when you hear about some service finding hundreds of vulnerabilities with their super new AI security tool, that's impressive, but the actually impressive part is if they are coordinating the findings. Because the tool probably took an hour or two but the coordination is going to take 10 to 100 times that much time.
-
I keep seeing stories about LLMs finding vulnerabilities. Finding vulnerabilities was never the hard part, the hard part is coordinating the disclosure
It looks like LLMs can find vulnerabilities at an alarming pace. Humans aren't great at this sort of thing, it's hard to wade through huge codebases, but there are people who have a talent for vulnerability hunting.
This sort of reminds me of the early days of fuzzing. I remember fuzzing libraries and just giving up because they found too many things to actually handle. Eventually things got better and fuzzing became a lot harder. This will probably happen here too, but it will take years.
What about this coordinating thing?
When you find a security vulnerability, you don't open a bug and move on. You're expected to handle it differently. Even before you report it, you need at a minimum a good reproducer and explanation of the problem. It's also polite to write a patch. These steps are difficult, maybe LLMs can help, we shall see.
Then you contact a project, every project will have a slightly different way they like to have security vulnerabilities reported. You present your evidence and see what happens. It's very common for some discussion to ensue and patch ideas to evolve. This can take days or even weeks. Per vulnerability.
So when you hear about some service finding hundreds of vulnerabilities with their super new AI security tool, that's impressive, but the actually impressive part is if they are coordinating the findings. Because the tool probably took an hour or two but the coordination is going to take 10 to 100 times that much time.
@joshbressers newer tools have trended towards customization (AFL++, Jackalope for fuzzing, CodeQL for static analysis) because vulnerability research requires knowledge of the code and environment to have effective results.
I worry for open source having to deal with a worse version of "my scanner found a thousand uses of strcpy", and having to divert attention from actual security issues.
-
R relay@relay.infosec.exchange shared this topicR relay@relay.an.exchange shared this topic
-
I incentive moved this topic from Uncategorized
