There used to be a time when building out a botnet required *some* work – writing exploits, taking over devices, obscuring the purpose of the executable, etc.
-
@rysiek Yeah. Remember when "please forward this virus to your friends" was a dry joke?
@jmax "and delete your files"
-
It is also entirely par for the course for the broader "AI" ecosystem, which has the same scammy vibes as the NFT space.
For years Microsoft had a line in Copilot's ToS (still does) insisting it is for entertainment purposes only (yet they push it in their products):
https://www.theregister.com/2024/08/14/microsoft_services_agreement_update_warns/Anthropic's "extensively trained" model got tricked by a tactic used by a 13yo – "really, I'm a researcher!" and the company still does not see it as their responsibility:
https://rys.io/en/181.html#ai-orchestrated-cyberattack
🧵/end
@rysiek considering the peeks into the leaked Claude Code, jailbreaking it this way is explicitly allowed in the code itself. If you tell it you are part of a security research team or on an authorized entertainment or doing a computer security assignment, it will let you do what you want.
-
@rysiek considering the peeks into the leaked Claude Code, jailbreaking it this way is explicitly allowed in the code itself. If you tell it you are part of a security research team or on an authorized entertainment or doing a computer security assignment, it will let you do what you want.
@GreatBigTable interesting. I have not dove into Claude Code's spaghetti myself. Would love to hear more about this.
-
There used to be a time when building out a botnet required *some* work – writing exploits, taking over devices, obscuring the purpose of the executable, etc.
Not any more!
Instead of "malware", call it an "AI agent" and people will just happily install it on their devices with full root privileges!
https://github.com/jgamblin/OpenClawCVEs/Bam! RCE by asking nicely.
🧵
@rysiek Should we... move back into caves now, and get a head start on this utter collapse?..
-
There used to be a time when building out a botnet required *some* work – writing exploits, taking over devices, obscuring the purpose of the executable, etc.
Not any more!
Instead of "malware", call it an "AI agent" and people will just happily install it on their devices with full root privileges!
https://github.com/jgamblin/OpenClawCVEs/Bam! RCE by asking nicely.
🧵
@rysiek Go fast and break stuff
-
@rysiek Should we... move back into caves now, and get a head start on this utter collapse?..
@cauZation I reject the false dichotomy of "OpenClaw or caves." We can make sane technological choices without rejecting modern technology completely. We do have that agency.
-
@cauZation I reject the false dichotomy of "OpenClaw or caves." We can make sane technological choices without rejecting modern technology completely. We do have that agency.
-
@cauZation those luxury bunkers are not going to actually help them all that much. They are the super-rich equivalent of fidgeting to deal with existential anxiety.
And of course status symbols.
-
OpenClaw is utterly negligent in promoting their stuff to regular users and not having gigantic warnings on their landing page and installation guides.
Their response to these vulnerabilities, mentioning 128 advisories that are "still pending assignment", and shilling their "managed" service, is laughable and craven.
And the way they hide behind the open source label is infuriating:
> The open-source model means every vulnerability gets public scrutiny and transparent fixes.
🧵
@rysiek this is absolutely true
It is also true that you can't stop a creek with your hand
-
@cauZation those luxury bunkers are not going to actually help them all that much. They are the super-rich equivalent of fidgeting to deal with existential anxiety.
And of course status symbols.
@rysiek I used to fully believe that, until a peer in the Gen 4 nuclear industry told me why the wealthiest are investing in it. And some of them are indulging certain crypto financing because it more directly ties to data centers and cornering energy markets.
They're making a massive leap to control as much vital resources as they possibly can, and have perhaps the most subservient subjugation in modern history.
-
@rysiek I used to fully believe that, until a peer in the Gen 4 nuclear industry told me why the wealthiest are investing in it. And some of them are indulging certain crypto financing because it more directly ties to data centers and cornering energy markets.
They're making a massive leap to control as much vital resources as they possibly can, and have perhaps the most subservient subjugation in modern history.
@cauZation oh no doubt. But if shit really hits the fan, the end result will be an unlivable planet, and at that stage the bunkers are only going to extend the agony.
-
@cauZation oh no doubt. But if shit really hits the fan, the end result will be an unlivable planet, and at that stage the bunkers are only going to extend the agony.
@rysiek lol, I tell myself that every I eat fresh protein, and doubt canned food can help reemerge a superior, even more abusive power from said bunkers
-
@davidgerard ah, sorry! Fixing. The broader point stands.
@rysiek oh it's absolutely clown shoes
but also this is only the consumer terms
so in many countries, advertising laws kick in! if they don't state the gotcha right there in the ads, they risk a finding of deceptive practices
-
Do they mention any of this on their landing page? No, of course not:
https://openclawai.io/Do they mention this on their quickstart page? No, of course not:
https://openclawai.io/quickstartBut they sure mention the managed hosting that is "coming soon"! Which of course they shill in their blogpost about the vulnerabilities:
> For many users, that’s a reasonable tradeoff. For others, it’s the argument for managed hosting.
Security fuckup? More like business opportunity, amirite?
🧵
@rysiek I love how this landed right after the trive/litellm/axios thing where the take out is the exact opposite of “upgrade as soon as there is a new version”.
-
@rysiek I love how this landed right after the trive/litellm/axios thing where the take out is the exact opposite of “upgrade as soon as there is a new version”.
@marcink right?
Don't worry, as soon as OpenClaw gets hit by supply-chain attack, which they inevitably are going to, this will flip.
-
@GreatBigTable interesting. I have not dove into Claude Code's spaghetti myself. Would love to hear more about this.
I guess you have to ask really, really nicely, to counteract the other instruction. Or simply add a "system reminder".
From a great and very enjoyable thread (for certain subcategories of "enjoyable"):
jonny (good kind) (@jonny@neuromatch.social)
Attached: 3 images i love this. there's a mechanism to slip secret messages to the LLM that it is told to interpret as system messages. there is no validation around these of any kind on the client, and there doesn't seem to be any differentiation about location or where these things happen, so that seems like a nice prompt injection vector. this is how claude code reminds the LLM to not do a malware, and it's applied by just string concatenation. i can't find any place that gets stripped aside from when displaying output. it actually looks like all the system reminders get catted together before being send to the API. neat!
neurospace.live (neuromatch.social)
-
I guess you have to ask really, really nicely, to counteract the other instruction. Or simply add a "system reminder".
From a great and very enjoyable thread (for certain subcategories of "enjoyable"):
jonny (good kind) (@jonny@neuromatch.social)
Attached: 3 images i love this. there's a mechanism to slip secret messages to the LLM that it is told to interpret as system messages. there is no validation around these of any kind on the client, and there doesn't seem to be any differentiation about location or where these things happen, so that seems like a nice prompt injection vector. this is how claude code reminds the LLM to not do a malware, and it's applied by just string concatenation. i can't find any place that gets stripped aside from when displaying output. it actually looks like all the system reminders get catted together before being send to the API. neat!
neurospace.live (neuromatch.social)
@wakame @GreatBigTable ah yes, I've seen that in fact
-
@wakame @GreatBigTable ah yes, I've seen that in fact
-
@GreatBigTable @wakame indeed, somehow I missed that initially. Thanks!
-
There used to be a time when building out a botnet required *some* work – writing exploits, taking over devices, obscuring the purpose of the executable, etc.
Not any more!
Instead of "malware", call it an "AI agent" and people will just happily install it on their devices with full root privileges!
https://github.com/jgamblin/OpenClawCVEs/Bam! RCE by asking nicely.
🧵
@rysiek
If I were ever interested in experimenting with that kind of thing (I'm not), I would only do it in a virtual machine. To do otherwise is foolish.
