It's amazing how fast attitudes to security in the industry has changed.
-
hacking a computer program pretending to be a human is like some weird neo-victorian parlor game in The Diamond Age
@foone it has also passed through our mind that if there is ever some really high-stakes version in which we have to do so, for survival's sake, our ability to do it will depend on the extent to which our personal way of thinking is under-represented in the training set. so we're not eager to donate our efforts to anyone's training set.
-
You forgot the part where a developer deletes production database and its all volume backups via an agent in 9 seconds, and forces the agent to confess the error.
Like the agent has its own mind.
Unbelievable.
@bayindirh @foone can't have vulnerabilities without any production!
-
You forgot the part where a developer deletes production database and its all volume backups via an agent in 9 seconds, and forces the agent to confess the error.
Like the agent has its own mind.
Unbelievable.
@bayindirh yeah that story (and some recent experiences I can't go into) is what prompted this
-
@bayindirh @foone can't have vulnerabilities without any production!
-
@foone I do wonder how certain industries and institutions are doing under the pressure to conform to these new ways of doing things (banks, hospitals, scientists)
@foone I used to work for a bank with a huge security overhead. The machines that everyone used had quite a few limitations due to security. Windows 11 w Copilot must have them like the Chihuahua from Ren & Stimpy
-
It's amazing how fast attitudes to security in the industry has changed. Like, I remember in 2023ish spending a while working on a system to securely trigger remote builds, because we couldn't have our slack chatbots on the same network as our Jenkins server
And in 2026 they just give a 3rd party LLM write access to both + the git repo
@foone but on the other hand the regulators still haven't read the correct horse battery staple XKCD cartoon and still demand stupid passwords that everyone forgets.
-
It's amazing how fast attitudes to security in the industry has changed. Like, I remember in 2023ish spending a while working on a system to securely trigger remote builds, because we couldn't have our slack chatbots on the same network as our Jenkins server
And in 2026 they just give a 3rd party LLM write access to both + the git repo
@foone How long until I can find sensitive government & corporate computers exposed to the Internet by wardialing again? Maybe malicious actors can start asking company chatbots to open telnet ports.
-
It's amazing how fast attitudes to security in the industry has changed. Like, I remember in 2023ish spending a while working on a system to securely trigger remote builds, because we couldn't have our slack chatbots on the same network as our Jenkins server
And in 2026 they just give a 3rd party LLM write access to both + the git repo
@foone people stopped caring. this is what 'work alienation' does to 'the work'. it removes the craft, the skill. workers are swapped out like Legos before they become invested in the work, before they become expensive.
the quality stops mattering, because the company will stop existing in 5 years, when guarantees turn into lawsuits.
funds are received to start projects, but every one leaves before the finish. no one is responsible. no one cares.
-
@foone I used to work for a bank with a huge security overhead. The machines that everyone used had quite a few limitations due to security. Windows 11 w Copilot must have them like the Chihuahua from Ren & Stimpy
-
It's amazing how fast attitudes to security in the industry has changed. Like, I remember in 2023ish spending a while working on a system to securely trigger remote builds, because we couldn't have our slack chatbots on the same network as our Jenkins server
And in 2026 they just give a 3rd party LLM write access to both + the git repo
@foone Just like a lot of other stuff, once they didn't have to make an effort to care they immediately stopped.
-
It's amazing how fast attitudes to security in the industry has changed. Like, I remember in 2023ish spending a while working on a system to securely trigger remote builds, because we couldn't have our slack chatbots on the same network as our Jenkins server
And in 2026 they just give a 3rd party LLM write access to both + the git repo
@foone the ai companies present it all as a neck or nothing kind of thing. And that horrifies me. I used to be the CTO for a federal contractor. We did facilities management. And I could never imagine a fairly independent program having access to say our contracts, some of which were for classified projects. If you were an OpenAI sales rep and proposed that to me, you would be escorted out of my office. But people are doing it!!! For some goddammed unknown reason.
-
@foone the ai companies present it all as a neck or nothing kind of thing. And that horrifies me. I used to be the CTO for a federal contractor. We did facilities management. And I could never imagine a fairly independent program having access to say our contracts, some of which were for classified projects. If you were an OpenAI sales rep and proposed that to me, you would be escorted out of my office. But people are doing it!!! For some goddammed unknown reason.
-
-
@foone I love this sort of stuff tbh. Just like NFTs, it's great to have a filter like this that clearly shows who's actually nuts and who isn't.
-
hacking a computer program pretending to be a human is like some weird neo-victorian parlor game in The Diamond Age
@foone thankfully, we all agree that it would be a terrible idea to make anything from Neil Stephenson books real.
…right?
-
@foone thankfully, we all agree that it would be a terrible idea to make anything from Neil Stephenson books real.
…right?
@foone …sorry, it just hit me that they've done *three* of them now. Cryptonomicon, Snow Crash, The Diamond Age: crypto, metaverse, AI.
Why is our society run by people who think the Torment Nexus sounds neat?
-
if I can convince your chatbox to add a new dependency to your software and push a new version to prod, it's just not worth my time to bother
@foone@digipres.club at this point i just laugh at the absurdity of it all, companies basically scrambling to put the world's most trusting doorman in front of all their sensitive tech and all you need now is a "pwetty pwease
"
-
I have SEPARATE TOOLS and TECHNIQUES for hacking humans and computer hardware and computer software. Mixing them up is just wrong and unfun.
@foone Alice Averlong, Authorized and Certified Gender Technician.
-
@foone Alice Averlong, Authorized and Certified Gender Technician.
@Jenetrix nice!
-
@Jenetrix nice!
@Jenetrix@shrimp.creatures.club @foone@digipres.club
it is kind of amazing how chill the industry is with giving models designed to generate and run unauditable code based on arbitrary unsanitized user input access to… well anything, really
-
R relay@relay.mycrowd.ca shared this topic
