@adamshostack how is your experience with using LLMs or agentic AI for threat modeling?
-
@adamshostack how is your experience with using LLMs or agentic AI for threat modeling? I have so far tried GPT5 for creating attack trees and to find STRIDE threats from a textual prompt describing the goal, setup, elements etc. The results were good enough to use as a starting point for further refinement. What I'm wondering now is, if I could achieve similar results using a self hosted, smaller ai model and feed it a well structured draw.io diagram of the environment containing the elements, data flows and trust boundaries instead of a free textual prompt. Has anyone done this before or are you aware of any ready to go tools for self-hosting that can do that? I don't want to feed public LLMs any information about what threat models I want to create.
-
@adamshostack how is your experience with using LLMs or agentic AI for threat modeling? I have so far tried GPT5 for creating attack trees and to find STRIDE threats from a textual prompt describing the goal, setup, elements etc. The results were good enough to use as a starting point for further refinement. What I'm wondering now is, if I could achieve similar results using a self hosted, smaller ai model and feed it a well structured draw.io diagram of the environment containing the elements, data flows and trust boundaries instead of a free textual prompt. Has anyone done this before or are you aware of any ready to go tools for self-hosting that can do that? I don't want to feed public LLMs any information about what threat models I want to create.
@d3tm4r I've blogged on this extensively, for example https://shostack.org/blog/lessons-from-owasp/ (and dig through the AI category) I haven't dug at all into the local/frontier model tradeoffs.
-
@d3tm4r I've blogged on this extensively, for example https://shostack.org/blog/lessons-from-owasp/ (and dig through the AI category) I haven't dug at all into the local/frontier model tradeoffs.
@adamshostack I have read some of your blog articles on the topic. Thanks for pointing me to them. I see now that despite my general skepticism I was still too enthusiastic about the first results that I got out of ChatGPT.
I am teaching threat modeling for beginners for a few years now and established threat modeling in our organization's quality gates but still adoption is far below what I think is needed. So I was hoping to make it easier for people by means of automation and since LLMs are the rage it is worth a try. But results have to be of consistent quality and quantity even if they should be regarded as a starting point only.
To be honest, I'd prefer deterministic tools that can leverage threat libraries and frameworks and take a DFD or architecture diagram as input. However I haven't found any good tools for self hosting so far. The time that I have for this is very limited though since my main job is being a SOC manager nowadays. -
@adamshostack @tychotithonus a new day, a new shot at using ChatGPT for threat modeling.
It did a pretty decent job identifying elements, data flows and trust boundaries from my draw.io DFD but it listed very few threats in the beginning.
I then promoted it to find more threats, focus on the element that has to be protected and use threats from the Microsoft EoP card game, which resulted in some more threats. Have to review the results more thoroughly later though.
Here's a gist: https://gist.github.com/test4bounty/7d78a5fca56645db6ca2e3d7193525a5 -
R relay@relay.infosec.exchange shared this topic
-
@adamshostack @tychotithonus a new day, a new shot at using ChatGPT for threat modeling.
It did a pretty decent job identifying elements, data flows and trust boundaries from my draw.io DFD but it listed very few threats in the beginning.
I then promoted it to find more threats, focus on the element that has to be protected and use threats from the Microsoft EoP card game, which resulted in some more threats. Have to review the results more thoroughly later though.
Here's a gist: https://gist.github.com/test4bounty/7d78a5fca56645db6ca2e3d7193525a5@adamshostack @tychotithonus doesn't look like threats from the EoP card game were in the training data.
