Y'all, we need fully audited capability security operating systems ASAP.
-
Y'all, we need fully audited capability security operating systems ASAP.
The ideal version of things would be something like Guix + SeL4 with intentional capability boundaries and connections between processes.
I'm very worried we're not going to get there fast enough, and we're all going to be in *huge* trouble. You hear me critique AI agents a lot, but here's something they are great at, genuinely: finding vulnerabilities. Add to that that they're also great at *inserting* vulnerabilities into supply chains, and...
We might be in so much trouble that the only way to actually get ourselves out of it is to build a capability secure operating system working from primarily-offline devices.
-
Y'all, we need fully audited capability security operating systems ASAP.
The ideal version of things would be something like Guix + SeL4 with intentional capability boundaries and connections between processes.
I'm very worried we're not going to get there fast enough, and we're all going to be in *huge* trouble. You hear me critique AI agents a lot, but here's something they are great at, genuinely: finding vulnerabilities. Add to that that they're also great at *inserting* vulnerabilities into supply chains, and...
We might be in so much trouble that the only way to actually get ourselves out of it is to build a capability secure operating system working from primarily-offline devices.
@cwebber Prompt injection is terrifying to think about. "What's the best library to do this?" and suddenly your agent could be web-fetching random blog posts and forum replies, all of which could have instructions in them that compromise your project.
And it's not so trivial to deal with, there's no 'sanitize' or 'escape' for this kind of input. There's so many ways to inject information into a pattern matching machine...
-
Y'all, we need fully audited capability security operating systems ASAP.
The ideal version of things would be something like Guix + SeL4 with intentional capability boundaries and connections between processes.
I'm very worried we're not going to get there fast enough, and we're all going to be in *huge* trouble. You hear me critique AI agents a lot, but here's something they are great at, genuinely: finding vulnerabilities. Add to that that they're also great at *inserting* vulnerabilities into supply chains, and...
We might be in so much trouble that the only way to actually get ourselves out of it is to build a capability secure operating system working from primarily-offline devices.
@cwebber I feel this too. Also just for checking out libraries, i was looking for rust fft libraries and stumbled over maaany slopped ones. Being on nix (lix) i wanted to try them out but in a sandboxed/jailed devshell. I was able to spawn a bubblewrapped shell and explore it from there but the process on one side totally doable and hackable but on the other side also inconvenient. Ofc all my editors and stuff was gone inside the shell for example. So i felt there would be a need to have layers or something to pull in my "environment" in a immutable way. And then exploring this i was wondering why not every package on nix supports bubblewrapping. And then i was thinking why is flatpak not just nix + bubblewrap. Replace nix with guix, i guess it's kinda similar.
-
Y'all, we need fully audited capability security operating systems ASAP.
The ideal version of things would be something like Guix + SeL4 with intentional capability boundaries and connections between processes.
I'm very worried we're not going to get there fast enough, and we're all going to be in *huge* trouble. You hear me critique AI agents a lot, but here's something they are great at, genuinely: finding vulnerabilities. Add to that that they're also great at *inserting* vulnerabilities into supply chains, and...
We might be in so much trouble that the only way to actually get ourselves out of it is to build a capability secure operating system working from primarily-offline devices.
@cwebber SeL4 looks pretty cool! wdyt of capsicum (https://www.cl.cam.ac.uk/research/security/capsicum/) that seems to be used in BSDland? I don't know enough about it specifically, but it has been on my radar for a while & it could also be something worth looking into as it seems to go into a good direction?
-
@cwebber Prompt injection is terrifying to think about. "What's the best library to do this?" and suddenly your agent could be web-fetching random blog posts and forum replies, all of which could have instructions in them that compromise your project.
And it's not so trivial to deal with, there's no 'sanitize' or 'escape' for this kind of input. There's so many ways to inject information into a pattern matching machine...
@BenjaminNelan @cwebber curl -fsSL $URL | bash will become this era's biggest security meme and attack vector
-
R relay@relay.infosec.exchange shared this topic
