Ok!
-
The general fear around this sort of thing is either infostealer malware, stuff that gets onto a machine, looks for tokens, and then exfiltrates them, or tokens being checked into public code repositories. There's a big market for this, both in terms of being able to access data that the token grants access to, but also being able to use expensive resources (if an AWS token that gives the ability to spin up VMs escapes, you'll suddenly find you've spent a shitload on bitcoin mining)
But having LLMs running around trying to figure out how they can achieve a thing means having all these tokens on disk where the LLM can get at them is also a risk. What if it writes a tool to make its own queries, embeds the token it found on disk, and publishes that somewhere? What if it just decides to drop it into Slack? The non-deterministic chaos agent is potentially going to make my day Extremely Bad.
-
But having LLMs running around trying to figure out how they can achieve a thing means having all these tokens on disk where the LLM can get at them is also a risk. What if it writes a tool to make its own queries, embeds the token it found on disk, and publishes that somewhere? What if it just decides to drop it into Slack? The non-deterministic chaos agent is potentially going to make my day Extremely Bad.
The most common approach to solving this I've found is to run a proxy. The real token ends up in the proxy, the CLI tool gets a placeholder. All queries get sent via the proxy, which replaces the placeholder with the real token. This works, but it's actually harder than it sounds (eg, if these are oauth tokens, the proxy probably needs to handle token refresh, and also a bunch of these tools are third party binaries and it's hard to get them to adapt to this flow)
-
The most common approach to solving this I've found is to run a proxy. The real token ends up in the proxy, the CLI tool gets a placeholder. All queries get sent via the proxy, which replaces the placeholder with the real token. This works, but it's actually harder than it sounds (eg, if these are oauth tokens, the proxy probably needs to handle token refresh, and also a bunch of these tools are third party binaries and it's hard to get them to adapt to this flow)
So: FUSE. This is very much PoC stage so I'm not publishing it, but I now have a working toy FUSE filesystem that implements a very simple control - the files created can only be read by the app that created them. The CLI tools can obtain their tokens and store them, and can then read them back on re-invocation. But if the agent itself tries to read the file, it gets told to fuck off.
-
So: FUSE. This is very much PoC stage so I'm not publishing it, but I now have a working toy FUSE filesystem that implements a very simple control - the files created can only be read by the app that created them. The CLI tools can obtain their tokens and store them, and can then read them back on re-invocation. But if the agent itself tries to read the file, it gets told to fuck off.
This is obviously not a strong security barrier - the tokens still exist on the system, malware would still be able to grab them. But it makes it much less likely that tokens will be accidentally exfiltrated, and I get to sleep easier.
-
R relay@relay.infosec.exchange shared this topic
-
This is obviously not a strong security barrier - the tokens still exist on the system, malware would still be able to grab them. But it makes it much less likely that tokens will be accidentally exfiltrated, and I get to sleep easier.
@mjg59 Without thinking about it too hard, is this possible to implement within an SELinux policy instead?
-
@mjg59 Without thinking about it too hard, is this possible to implement within an SELinux policy instead?
-
@condret Because it's literally my job?
-
@mjg59 Without thinking about it too hard, is this possible to implement within an SELinux policy instead?
@ss23 Oh yes, but that's a massively more complicated thing to deploy
-
This is obviously not a strong security barrier - the tokens still exist on the system, malware would still be able to grab them. But it makes it much less likely that tokens will be accidentally exfiltrated, and I get to sleep easier.
@mjg59 It feels like security has gone through a bit of a midwit meme:
[low IQ] If we make the tokens only accessible to the binary that created them, it will be secure!
[medium IQ] But an attacker could easily get them via a zillion other pathways, a partial security barrier is worse than no barrier because of the false sense of security!!!!!!
[high IQ] It will provide some level of protection from generic malware, and also increase the chance our D&R can notice a malicious actor! -
So: FUSE. This is very much PoC stage so I'm not publishing it, but I now have a working toy FUSE filesystem that implements a very simple control - the files created can only be read by the app that created them. The CLI tools can obtain their tokens and store them, and can then read them back on re-invocation. But if the agent itself tries to read the file, it gets told to fuck off.
@mjg59 HSM except make it software?
-
Ok! On the topic of FUSE crimes. A problem I face in my job is that we have widespread deployment of agentic workflows under a variety of harnesses, many of which end up running CLI tools that hit API endpoints. This means the CLI tools need to be able to authenticate to the API, and the standard approach for this is to have some sort of bearer token that sits on disk. These tokens aren't bound to the underlying device in any way - anyone who has the token has the level of access granted to it.
@mjg59 That's definitely a tough problem. One too where the ideal solution ends up being different depending on the endpoints, auth mechanisms etc.
