<![CDATA[Incredible.]]>Incredible. Every second paragraph in this article is lunatic nonsense.

One of the things I've long said about hiring is that you can always tell when you're talking to a junior dev who's going to be senior-staff or better someday. You can always tell when somebody was paying attention in the theory classes.

But good god you can also tell when people missed that day in gradeschool when somebody slowly went over "So, what is a computer, really."

archive.ph

favicon

(archive.ph)

]]>https://board.circlewithadot.net/topic/b7703b14-5eda-4610-ba49-fc49313972f4/incredible.RSS for NodeFri, 01 May 2026 07:22:47 GMTMon, 27 Apr 2026 16:56:43 GMT60<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 18:37:06 GMT]]>@henryk @adamshostack @mhoye I had the exact same feeling/memory

]]>https://board.circlewithadot.net/post/https://chaos.social/users/vollkorn/statuses/116478117595715351https://board.circlewithadot.net/post/https://chaos.social/users/vollkorn/statuses/116478117595715351Mon, 27 Apr 2026 18:37:06 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 18:30:55 GMT]]>@mwl @mhoye

While trying to keep my eyebrows from physically leaving my head, I imagined little speech bubbles as I read that article:

"My business was deleted by the best premium model running on the finest of agents."

"Screw you, Poindexter, I don't need to know what a context window is."

"Claude's not stochastic, *you're* stochastic."

]]>https://board.circlewithadot.net/post/https://dju.social/users/eswag/statuses/116478093262328299https://board.circlewithadot.net/post/https://dju.social/users/eswag/statuses/116478093262328299Mon, 27 Apr 2026 18:30:55 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 18:29:10 GMT]]>@mhoye and for some companies it will be less bad because there's a measure of defense in depth, such as keeping actual backups. But the more decision-makers push this nonsense in every nook and cranny of the tech world, the closer will be to unrecoverable failure cascades.

]]>https://board.circlewithadot.net/post/https://mas.to/users/gabrielesvelto/statuses/116478086380598727https://board.circlewithadot.net/post/https://mas.to/users/gabrielesvelto/statuses/116478086380598727Mon, 27 Apr 2026 18:29:10 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 18:27:55 GMT]]>@mhoye I think one important aspect here is that's not just this company. This is the entire industry you're looking at. Practically every decision maker who has little or no knowledge of tech is now fully on board this hype train. Every one of them and every company they work for is a prompt away from doing something unbelievably stupid and possibly fatal.

]]>https://board.circlewithadot.net/post/https://mas.to/users/gabrielesvelto/statuses/116478081434159602https://board.circlewithadot.net/post/https://mas.to/users/gabrielesvelto/statuses/116478081434159602Mon, 27 Apr 2026 18:27:55 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 18:18:20 GMT]]>@henryk @adamshostack @mhoye only 2 deleted production servers a year, I can't wait for the model to improve and get to 1 deleted production server a month, I'm sure AI will get there by the end of this year...

]]>https://board.circlewithadot.net/post/https://hachyderm.io/users/fcbsd/statuses/116478043754250080https://board.circlewithadot.net/post/https://hachyderm.io/users/fcbsd/statuses/116478043754250080Mon, 27 Apr 2026 18:18:20 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 18:17:52 GMT]]>@mhoye this is just … exactly the replit thing again, isn't it? from last year? https://www.pcmag.com/news/vibe-coding-fiasco-replite-ai-agent-goes-rogue-deletes-company-database

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/glyph/statuses/116478041949208538https://board.circlewithadot.net/post/https://mastodon.social/users/glyph/statuses/116478041949208538Mon, 27 Apr 2026 18:17:52 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:51:29 GMT]]>(Credit for the inspiration, where it's belongs, this is me riffing on Avery Edison's razor-sharp tweet from a few years ago)

Link Preview Image
]]>https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477938203192457https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477938203192457Mon, 27 Apr 2026 17:51:29 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:48:55 GMT]]>@henryk @mhoye It's not opening on my device, but the "This isn't a story about one bad agent or one bad API. It's about an entire industry ..." quoted above already had my slop sense tingling.

]]>https://board.circlewithadot.net/post/https://mastodon.sdf.org/users/DamonWakes/statuses/116477928119945646https://board.circlewithadot.net/post/https://mastodon.sdf.org/users/DamonWakes/statuses/116477928119945646Mon, 27 Apr 2026 17:48:55 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:41:35 GMT]]>I wrote the words "I confess, I did it, I take full responsibility" on a piece of paper. I was ready to turn myself in, to atone for my crimes. But then I put that piece of paper in a photocopier, and when I pressed the green button I learned something amazing. And what a weight off my conscience! The only question was, how did the photocopier manage to poison the Widow Bentley, drive over Baron Grimald, push the Duchess of Lockley out the balcony window and still manage to frame the butler?

]]>https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477899289530408https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477899289530408Mon, 27 Apr 2026 17:41:35 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:41:18 GMT]]>@mhoye Don't worry, I'm pretty sure the text is extruded, too. I've never seen a "The pattern is clear." in a context like this on human text, but am encountering it unreasonably often in LLM generated text.

]]>https://board.circlewithadot.net/post/https://chaos.social/users/henryk/statuses/116477898183954600https://board.circlewithadot.net/post/https://chaos.social/users/henryk/statuses/116477898183954600Mon, 27 Apr 2026 17:41:18 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:37:03 GMT]]>@mhoye I fear that the big enterprise takeaway from this story will be “our controls and guardrails are much better than that”.

]]>https://board.circlewithadot.net/post/https://lingo.lol/ap/users/116404198720403352/statuses/116477881472893010https://board.circlewithadot.net/post/https://lingo.lol/ap/users/116404198720403352/statuses/116477881472893010Mon, 27 Apr 2026 17:37:03 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:30:28 GMT]]>But my favourite part of this, bar none, is how it's everyone else's fault.

It's Cursor's fault, Railway's fault, maybe even Anthropic's fault, someone's gonna hear from my lawyer.

The CEO of a company running a stochastic stack without access control, data hygiene or backups is blameless and powerless. That's AI's real selling point, after all: It's Not My Fault As A Service.

"This isn't a story about one bad agent or one bad API. It's about an entire industry ..."

Or, maybe it's you.

]]>https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477855549124422https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477855549124422Mon, 27 Apr 2026 17:30:28 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:28:11 GMT]]>@adamshostack @mhoye I'm confused. I had to check the date. I am *very* sure I read the "the LLM deleted my prod and when confronted, it confessed!" story before. Roughly 6 months ago, maybe a year.

Ahh, here it is: https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/

]]>https://board.circlewithadot.net/post/https://chaos.social/users/henryk/statuses/116477846567974507https://board.circlewithadot.net/post/https://chaos.social/users/henryk/statuses/116477846567974507Mon, 27 Apr 2026 17:28:11 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:26:38 GMT]]>@mhoye That first paragraph: "This is the agent on record, in writing."

and herein lies the root of the failure: they actually believe that this is some sort of diagnostic, rather than just filling in a plausible response based on the question.

]]>https://board.circlewithadot.net/post/https://mstdn.social/users/darkling/statuses/116477840470491513https://board.circlewithadot.net/post/https://mstdn.social/users/darkling/statuses/116477840470491513Mon, 27 Apr 2026 17:26:38 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:21:05 GMT]]>@mhoye I’m so glad I didn’t study computer science, when that sort of knowledge clearly is no longer needed to run a software business

]]>https://board.circlewithadot.net/post/https://chaos.social/users/slothrop/statuses/116477818643197673https://board.circlewithadot.net/post/https://chaos.social/users/slothrop/statuses/116477818643197673Mon, 27 Apr 2026 17:21:05 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:20:34 GMT]]>@mhoye kek, I don't even need an LLM to accidentally all my Rails data. Many cycles ago, I ran wget --recursive against my cool little dev site, and didn't realize that it would also follow the "delete" links for all of the products I just entered. Bye bye data 🙃

]]>https://board.circlewithadot.net/post/https://weirder.earth/users/phred/statuses/116477816658491713https://board.circlewithadot.net/post/https://weirder.earth/users/phred/statuses/116477816658491713Mon, 27 Apr 2026 17:20:34 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:18:51 GMT]]>@mhoye There's a whole lotta YOLO in that story.

]]>https://board.circlewithadot.net/post/https://infosec.exchange/users/Sempf/statuses/116477809898716725https://board.circlewithadot.net/post/https://infosec.exchange/users/Sempf/statuses/116477809898716725Mon, 27 Apr 2026 17:18:51 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:16:36 GMT]]>@mhoye If only someone could invent some sort of, I dunno, approach or something that giving a single process all the power? authority? capabilities? privilege? was a bad thing, and we should go for less, not more.

]]>https://board.circlewithadot.net/post/https://infosec.exchange/users/adamshostack/statuses/116477801065248756https://board.circlewithadot.net/post/https://infosec.exchange/users/adamshostack/statuses/116477801065248756Mon, 27 Apr 2026 17:16:36 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:15:30 GMT]]>@mhoye I'm so glad that the "written confession" can't itself be hallucinated. That's a nice feature!

]]>https://board.circlewithadot.net/post/https://infosec.exchange/users/adamshostack/statuses/116477796716335742https://board.circlewithadot.net/post/https://infosec.exchange/users/adamshostack/statuses/116477796716335742Mon, 27 Apr 2026 17:15:30 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:12:17 GMT]]>@tito_swineflu @mhoye It's clowns all the way down.

]]>https://board.circlewithadot.net/post/https://hachyderm.io/users/dalias/statuses/116477784051234611https://board.circlewithadot.net/post/https://hachyderm.io/users/dalias/statuses/116477784051234611Mon, 27 Apr 2026 17:12:17 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:11:22 GMT]]>"The agent itself enumerates the safety rules it was given and admits to violating every one. This is not me speculating about agent failure modes. This is the agent on the record, in writing.

The "system rules" the agent is referring to are consistent with Cursor's documented system-prompt language and our project rules for this codebase. Both safeguards failed simultaneously."

What do you think is happening here? You know it's called a "language model", right? Did you ever wonder... why?

]]>https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477780484278451https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477780484278451Mon, 27 Apr 2026 17:11:22 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:07:31 GMT]]>@mhoye who the f publishes articles on that site...

It was rhetorical... AI bros do... Of course AI bros do...

]]>https://board.circlewithadot.net/post/https://social.petko.me/users/petko/statuses/116477765305182925https://board.circlewithadot.net/post/https://social.petko.me/users/petko/statuses/116477765305182925Mon, 27 Apr 2026 17:07:31 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:07:18 GMT]]>@mhoye I love that the first line in "What needs to change" isn't, "We should not let non-deterministic programs have free range across our systems"

]]>https://board.circlewithadot.net/post/https://sfba.social/users/tito_swineflu/statuses/116477764465210533https://board.circlewithadot.net/post/https://sfba.social/users/tito_swineflu/statuses/116477764465210533Mon, 27 Apr 2026 17:07:18 GMT<![CDATA[Reply to Incredible. on Mon, 27 Apr 2026 17:05:41 GMT]]>"Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it" WHAT IN THE WHAT, your full stack jenga provider does WHAT with BACKUPS WHAT my sweet summer child I know that legal jargon can be perplexing and counterintuitive at times but I feel like we all sort of understand that the word "due" in "due dilligence" means "more than none."

]]>https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477758123348493https://board.circlewithadot.net/post/https://cosocial.ca/users/mhoye/statuses/116477758123348493Mon, 27 Apr 2026 17:05:41 GMT