I wouldn't say that I enjoy these stories but I will say they certainly encourage me NOT to let "AI" anywhere anything I consider to be sensitive and/or valuable

[theowashere@mastodon.social]

@scalzi In germany we say: tja

[thriftwicker@mastodon.social]

@scalzi Neat!

[spzb@infosec.exchange]

@scalzi restoring from three-month old backups? Yeah I think I can see your problem and it’s not AI.

[pa27@mastodon.social]

@scalzi Interesting that they still treat the ai code as if it was a real person. Totally subsumed by anthropomorphism. Then, they create a narrative to blame Railway, not the decision to use an "ai" for critical stuff. This is, if you read accounts of such things, exactly what people do who have been scammed....

[babor@loud.computer]

@pattykimura @scalzi All a language model does is guess what text comes next. Putting "don't guess" in a prompt is like saying "don't predict the next token based on the tokens before".

[papa@wheeli.social]

@spzb

@scalzi

Yeah the "the backups were stored on the same volume as the production data" part is the "big" problem here. It works have happened eventually, AI just sped the disaster up a bit.

[sabik@rants.au]

@Plumbert @scalzi
I mean, it's not coding, is it; it's an interesting research project, with googly eyes stuck on, given production credentials

[ninestonesclose@mastodon.social]

@scalzi hilarious…. This will happen on a larger scale soon, because people in companies behave like this everywhere. It’s almost like AI is making people even dumber

[iveryam@mastodon.social]

@scalzi Claude said it was Russ Hanneman and his Tres Comas bottle.

[pattykimura@beige.party]

@babor @scalzi

And yet it states it guessed.

It's not sentient but it is given authority to act, as if it can only make choices (a,b,c...) within a logical limited universe, so the outcome is constrained and limited. After the all caps swear, it states it knew what it should have done within its programming direction, but did the opposite anyway. Because why?

[avacadoavenger@mastodon.social]

@scalzi Could have been worse. It could have been an authentication of a nuclear launch code. Not being sure what to do it could have allowed a launch order to proceed.