"AI can make mistakes, always check the results"

[matty@blahaj.zone]

@jenniferplusplus@hachyderm.io Also, I feel it just undermines LLM being actually useful if I had to manually search it up to verify it.

[flippac@types.pl]

@kerravonsen @emily_s @jenniferplusplus Not always: sometimes it's being used outside the design spec, sometimes that's because the design spec wasn't communicated clearly but not always, etc etc.

"When someone says 'computer error' rather than something more specific they're probably full of it" I'm fine with, but one of the realities of computing machines as opposed to the mathematical abstraction of computing is that like all machines they have a non-zero failure rate - even if it's pretty damn tiny.

Now, the amount of shite practice out there re error tolerance/resilience? Sure, we can talk about that (or skip it, because neither of us are newbies here). But bitflips absolutely happen in the wild, especially if someone didn't realise what it really took to keep their machine cool enough.

[flippac@types.pl]

@kerravonsen @emily_s @jenniferplusplus BCD existed: if I'm old enough to talk about FDIV I certainly remember the long buildup to Y2K (including everyone running into it while computing about the future)

[flippac@types.pl]

@kerravonsen @emily_s @jenniferplusplus The Epochalypse specifically is worse, mind: it's an entirely reasonable (initially implicit-spec) "holy shit we did not build this to work for that long and you did it anyway" problem that originated when the relevant software wasn't a piece of critical infrastructure.

For banks and the like, Y2K was expected long-term maintenance.

The epochalypse is, realistically, user error.

[n_dimension@infosec.exchange]

@mighty_orbot @jenniferplusplus

I would really love to live in your world.

Humans around me fuck up all the time.
Most of the time they will won't even apologise when they are sprung on their "hallucination"

And they don't come with a warning sticker

[gbargoud@masto.nyc]

@Crystal_Fish_Caves @jenniferplusplus

This does remind me of this fucking weirdness when buying a house:

Title insurance - Wikipedia

(en.wikipedia.org)

A lot of the US does not have the government keep track of who owns what land so when you buy a place, you need to also buy insurance that says that you are actually buying it from someone able to sell it.

As far as I can tell every other country just has a department that you can ask "hey is this the owner" and trust the answer.

[jenniferplusplus@hachyderm.io]

@kerravonsen hey just to be clear, you're doing it right now. You're saying the computer is permitted to be wrong. The consequences will land on whoever was able to avoid them, and they will deserve it for not getting out of the way

[daniel_blake@mastodon.top]

@jenniferplusplus

I think that being liable for the mistakes an AI that you use is only fair... They who live by the sword etc.

[soulsource@mastodon.gamedev.place]

@Daniel_Blake @jenniferplusplus The problems start if you aren't using the AI because you want to, but because you got ordered to use it.

Cory Doctorow has written a lot about what he calls Reverse Centaurs - persons having to work for a machine instead of persons using a machine. For instance:
https://pluralistic.net/2025/12/05/pop-that-bubble/#u-washington

[pidaroso@mastodon.social]

@jenniferplusplus s

[bredroll@mas.to]

@jenniferplusplus i agree, but I also think that LLMs being unreliable is part of the business model, if it gave acceptable answers first time you'd only ask one question, if it messes up slightly you type more stuff, you rephrase the prompt or rewrite the spec, all of which are more tokens that your org will actually pay for. its like builtin #enshittification from the start.

[waimeafalls@ohai.social]

@jenniferplusplus AI appears to "learn from its mistakes" and amplify them...

[keithjones@social.vivaldi.net]

@jenniferplusplus
AI is a nifty tool, but blindly trusting its output is foolish. AI should not be treated as an unquestionable authority, which I've personally see happen in the workplace. The novelty of AI makes it enjoyable for now, yet companies rushing to replace human experience and expertise with AI will soon see quality erode and trust vanish altogether. When that happens these companies will learn that once quality and trust are lost, winning them back is far harder than maintaining them.

[madengineering@mastodon.cloud]

@jenniferplusplus ai is the intern on seven tabs of acid. He can no longer tell the difference between truth and fiction, and this will lead to lots of mistakes, most of which will lead to you staring at your monitor in confusion.
You must at a minimum verify the work, make sure it corresponds to reality, and get ready to wtf.

[janxdevil@sfba.social]

@jenniferplusplus Note well: whether “you” are actually liable for the errors made in the output produce by AI in response to your prompting depends entirely on whether “you” are someone privileged with impunity for your own errors in judgment or you instead are someone accountable for forced errors outside your own control.

[kerravonsen@mastodon.au]

@jenniferplusplus I am quite confused as to how you concluded that I said that, when I've been pointing out that it is human error

[worik@mastodon.social]

@jenniferplusplus

LLMs do not make mistakes on their own, you make mistakes using them

> "AI can make mistakes, always check the results"

> I fucking loathe this phrase and everything that goes into it.

Why? It is good advice and important when using LLMs.

I use LLMs every day in my coding practice, and they do make errors (thank you compiler)

LLMs are a tool, and must be wielded. When you use them you are responsible for the results

[luc0x61@mastodon.gamedev.place]

@jenniferplusplus There's a misunderstanding, an "AI can" is like a "worms can", that's the subject. Now it all makes sense.

[srvanderplas@datavis.social]

@kerravonsen @emily_s @jenniferplusplus or a gamma ray and bit flip. But that should probably be caught.

[kerravonsen@mastodon.au]

@jenniferplusplus The computer is wrongly permitted to be wrong. I thought I was agreeing with you.