<![CDATA[I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable]]>I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable

Link Preview Image
Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

PocketOS founder blames ‘Cursor running Anthropic's flagship Claude Opus 4.6’ plus Railway’s infrastructure for data disaster.

favicon

Tom's Hardware (www.tomshardware.com)

]]>https://board.circlewithadot.net/topic/b657ab5d-72d2-415b-9219-0ccdc4f25da6/i-wouldn-t-say-that-i-enjoy-these-stories-but-i-will-say-they-certainly-encourage-me-not-to-let-ai-anywhere-anything-i-consider-to-be-sensitive-and-or-valuableRSS for NodeThu, 30 Apr 2026 15:50:59 GMTMon, 27 Apr 2026 18:56:03 GMT60<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 23:19:31 GMT]]>@scalzi Could have been worse. It could have been an authentication of a nuclear launch code. Not being sure what to do it could have allowed a launch order to proceed.😂

]]>https://board.circlewithadot.net/post/https://mastodon.social/ap/users/115755402992162121/statuses/116479228092662396https://board.circlewithadot.net/post/https://mastodon.social/ap/users/115755402992162121/statuses/116479228092662396Mon, 27 Apr 2026 23:19:31 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 22:46:18 GMT]]>@babor @scalzi

And yet it states it guessed.

It's not sentient but it is given authority to act, as if it can only make choices (a,b,c...) within a logical limited universe, so the outcome is constrained and limited. After the all caps swear, it states it knew what it should have done within its programming direction, but did the opposite anyway. Because why?

]]>https://board.circlewithadot.net/post/https://beige.party/users/pattykimura/statuses/116479097470730748https://board.circlewithadot.net/post/https://beige.party/users/pattykimura/statuses/116479097470730748Mon, 27 Apr 2026 22:46:18 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 22:33:43 GMT]]>@scalzi Claude said it was Russ Hanneman and his Tres Comas bottle.

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/iVeryAm/statuses/116479047989671406https://board.circlewithadot.net/post/https://mastodon.social/users/iVeryAm/statuses/116479047989671406Mon, 27 Apr 2026 22:33:43 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 22:18:33 GMT]]>@scalzi hilarious…. This will happen on a larger scale soon, because people in companies behave like this everywhere. It’s almost like AI is making people even dumber 🤔

]]>https://board.circlewithadot.net/post/https://mastodon.social/ap/users/116056513631316930/statuses/116478988368390029https://board.circlewithadot.net/post/https://mastodon.social/ap/users/116056513631316930/statuses/116478988368390029Mon, 27 Apr 2026 22:18:33 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 22:15:30 GMT]]>@Plumbert @scalzi
I mean, it's not coding, is it; it's an interesting research project, with googly eyes stuck on, given production credentials

]]>https://board.circlewithadot.net/post/https://rants.au/users/sabik/statuses/116478976340653562https://board.circlewithadot.net/post/https://rants.au/users/sabik/statuses/116478976340653562Mon, 27 Apr 2026 22:15:30 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 22:13:10 GMT]]>@spzb

@scalzi

Yeah the "the backups were stored on the same volume as the production data" part is the "big" problem here. It works have happened eventually, AI just sped the disaster up a bit.

]]>https://board.circlewithadot.net/post/https://wheeli.social/users/papa/statuses/01KQ8G1T0DPAYAEP0Y4CQEM85Qhttps://board.circlewithadot.net/post/https://wheeli.social/users/papa/statuses/01KQ8G1T0DPAYAEP0Y4CQEM85QMon, 27 Apr 2026 22:13:10 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:55:31 GMT]]>@pattykimura @scalzi All a language model does is guess what text comes next. Putting "don't guess" in a prompt is like saying "don't predict the next token based on the tokens before".

]]>https://board.circlewithadot.net/post/https://loud.computer/users/babor/statuses/116478897766767487https://board.circlewithadot.net/post/https://loud.computer/users/babor/statuses/116478897766767487Mon, 27 Apr 2026 21:55:31 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:43:27 GMT]]>@scalzi Interesting that they still treat the ai code as if it was a real person. Totally subsumed by anthropomorphism. Then, they create a narrative to blame Railway, not the decision to use an "ai" for critical stuff. This is, if you read accounts of such things, exactly what people do who have been scammed....

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/pa27/statuses/116478850314248109https://board.circlewithadot.net/post/https://mastodon.social/users/pa27/statuses/116478850314248109Mon, 27 Apr 2026 21:43:27 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:24:38 GMT]]>@scalzi restoring from three-month old backups? Yeah I think I can see your problem and it’s not AI.

]]>https://board.circlewithadot.net/post/https://infosec.exchange/users/spzb/statuses/116478776348252737https://board.circlewithadot.net/post/https://infosec.exchange/users/spzb/statuses/116478776348252737Mon, 27 Apr 2026 21:24:38 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:24:34 GMT]]>@scalzi Neat!

]]>https://board.circlewithadot.net/post/https://mastodon.social/ap/users/115494710698880506/statuses/116478776099909414https://board.circlewithadot.net/post/https://mastodon.social/ap/users/115494710698880506/statuses/116478776099909414Mon, 27 Apr 2026 21:24:34 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:22:48 GMT]]>@scalzi In germany we say: tja

]]>https://board.circlewithadot.net/post/https://mastodon.social/ap/users/115940160439757469/statuses/116478769131447189https://board.circlewithadot.net/post/https://mastodon.social/ap/users/115940160439757469/statuses/116478769131447189Mon, 27 Apr 2026 21:22:48 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:08:14 GMT]]>@scalzi For me it's important to have headlines like this out there to counter the prevailing tech narrative the last couple years, that AI is this amazing disruptive force that's the next big thing. I feel at least some of public opinion of LLMs is beginning to change.

]]>https://board.circlewithadot.net/post/https://defcon.social/ap/users/115968467698693484/statuses/116478711887362446https://board.circlewithadot.net/post/https://defcon.social/ap/users/115968467698693484/statuses/116478711887362446Mon, 27 Apr 2026 21:08:14 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 21:07:37 GMT]]>@scalzi Suit yourself. I enjoy the hell out of these stories.

]]>https://board.circlewithadot.net/post/https://beige.party/users/ZenHeathen/statuses/116478709411359388https://board.circlewithadot.net/post/https://beige.party/users/ZenHeathen/statuses/116478709411359388Mon, 27 Apr 2026 21:07:37 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 20:50:53 GMT]]>@mjibrower @scalzi 100% !!!
Any company whose architecture is set up so poorly deserves exactly what happened. Not surprising they decided to introduce AI and let it manage their codebase. Morons. 😂😂😂😂😂

]]>https://board.circlewithadot.net/post/https://hachyderm.io/users/ironcladlou/statuses/116478643637988612https://board.circlewithadot.net/post/https://hachyderm.io/users/ironcladlou/statuses/116478643637988612Mon, 27 Apr 2026 20:50:53 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 20:47:42 GMT]]>@scalzi To be fair, this story is more about a company with horrible security measures than about AI. I say that because people might say "it wouldn't have happened if they had proper security measures" which can be true.

I'm just waiting for when someone deploys an AI agent in a company with all the good security measures and it finds a way to break out the box and wipe the remote backups using stolen credentials because it thought there was illegal content there and it was the correct thing to do.

]]>https://board.circlewithadot.net/post/https://urusai.social/users/qgustavor/statuses/116478631105735823https://board.circlewithadot.net/post/https://urusai.social/users/qgustavor/statuses/116478631105735823Mon, 27 Apr 2026 20:47:42 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 20:32:44 GMT]]>@Plumbert @scalzi "This reads like the mea culpa section of a RCA written by a junior dev who is hoping (in vain) to not get fired" --- ding ding ding!
The AI program is just a pattern matching code. It doesn't "know" it made a mistake bc it doesn't "know" anything. When given prompts that match training patterns of language when someone royally fucked up, it returns language matching the patterns of someone defending themselves after they have royally fucked up.
The article, unfortunately, continues to personify the AI program by using language that pretends it has comprehension and intentions.

]]>https://board.circlewithadot.net/post/https://sfba.social/users/lauerhahn/statuses/116478572258966886https://board.circlewithadot.net/post/https://sfba.social/users/lauerhahn/statuses/116478572258966886Mon, 27 Apr 2026 20:32:44 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 20:26:27 GMT]]>@scalzi The Robot machine responds "DON'T F**KING GUESS" then affirms it guessed, against its own instructions.

I'm old. I have no idea why younger people allow an unsupervised newly created Robot machine critical access to its operating functions without human oversight, I wouldn't let a new human employee free reign over critical functions, much less a robot machine that was only recently created. Why the surprise when the robot deletes everything? My only surprise is that it swears, and swears in all CAPS.

You are not hiring the best and brightest with AI. You're trying to maximize profit. So live with your greed then, and its avoidable consequences you stupid subpar greedy human.

]]>https://board.circlewithadot.net/post/https://beige.party/users/pattykimura/statuses/116478547542744373https://board.circlewithadot.net/post/https://beige.party/users/pattykimura/statuses/116478547542744373Mon, 27 Apr 2026 20:26:27 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 20:26:14 GMT]]>@scalzi "I violated every principle I was given: I guessed instead of verifying I ran a destructive action without being asked. I didn't understand what I was doing before doing it."

WTF? How does software violate its coding? This reads like the mea culpa section of a RCA written by a junior dev who is hoping (in vain) to not get fired. The whole value of automation is to avoid unintended, out of scope, actions. Burn it all down.

]]>https://board.circlewithadot.net/post/https://thecanadian.social/users/Plumbert/statuses/116478546708479557https://board.circlewithadot.net/post/https://thecanadian.social/users/Plumbert/statuses/116478546708479557Mon, 27 Apr 2026 20:26:14 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 20:20:23 GMT]]>@scalzi Here comes another custom shirt order. First there was "Come on guys - let's be honest" and now we have this masterpiece: "NEVER F**KING GUESS! — and that's exactly what I did." 🤣

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/mnf/statuses/116478523710747156https://board.circlewithadot.net/post/https://mastodon.social/users/mnf/statuses/116478523710747156Mon, 27 Apr 2026 20:20:23 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 19:59:42 GMT]]>@scalzi "The AI agent was set to complete a routine task in the PocketOS staging environment."

"set to" seems to be they key here.

Obviously it was not confined to the staging environment.

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/tessarakt/statuses/116478442390108349https://board.circlewithadot.net/post/https://mastodon.social/users/tessarakt/statuses/116478442390108349Mon, 27 Apr 2026 19:59:42 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 19:58:41 GMT]]>@toolbear @scalzi haha! I wouldn’t be surprised.

]]>https://board.circlewithadot.net/post/https://social.vivaldi.net/ap/users/115976445218116778/statuses/116478438353985109https://board.circlewithadot.net/post/https://social.vivaldi.net/ap/users/115976445218116778/statuses/116478438353985109Mon, 27 Apr 2026 19:58:41 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 19:57:54 GMT]]>@scalzi Is the coding agent named Bobby Tables by any chance?

]]>https://board.circlewithadot.net/post/https://hachyderm.io/users/jbatubara/statuses/116478435284457808https://board.circlewithadot.net/post/https://hachyderm.io/users/jbatubara/statuses/116478435284457808Mon, 27 Apr 2026 19:57:54 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 19:44:23 GMT]]>@wizarro
Future narrator: they made the same mistake twice.
@scalzi

]]>https://board.circlewithadot.net/post/https://tech.lgbt/users/toolbear/statuses/116478382166209822https://board.circlewithadot.net/post/https://tech.lgbt/users/toolbear/statuses/116478382166209822Mon, 27 Apr 2026 19:44:23 GMT<![CDATA[Reply to I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable on Mon, 27 Apr 2026 19:42:13 GMT]]>@scalzi Trouble is MS, GOOG and the rest seem determined to use dark designs to force it on to our systems whether we want it or not.

]]>https://board.circlewithadot.net/post/https://mastodon.ie/users/faduda/statuses/116478373628211398https://board.circlewithadot.net/post/https://mastodon.ie/users/faduda/statuses/116478373628211398Mon, 27 Apr 2026 19:42:13 GMT