"AI can make mistakes, always check the results"
-
@kerravonsen @emily_s @jenniferplusplus BCD existed: if I'm old enough to talk about FDIV I certainly remember the long buildup to Y2K (including everyone running into it while computing about the future)
@kerravonsen @emily_s @jenniferplusplus The Epochalypse specifically is worse, mind: it's an entirely reasonable (initially implicit-spec) "holy shit we did not build this to work for that long and you did it anyway" problem that originated when the relevant software wasn't a piece of critical infrastructure.
For banks and the like, Y2K was expected long-term maintenance.
The epochalypse is, realistically, user error.
-
R relay@relay.mycrowd.ca shared this topicR relay@relay.infosec.exchange shared this topic
-
@jenniferplusplus Saying “AI can make mistakes” is exactly like saying “An adjustable rate mortgage can increase the interest rate at any time.” It’s not a question of “if”, but “how soon is it possible?”
@mighty_orbot @jenniferplusplus
I would really love to live in your world.
Humans around me fuck up all the time.
Most of the time they will won't even apologise when they are sprung on their "hallucination"And they don't come with a warning sticker
-
@jenniferplusplus right?! What else would you buy if right on the lable it said "this may not be what we say it is" ??
So it may not be correct information, you don't know which part. You are using it to not have to do the legwork yourself. Do you
Take what it gave you, fingers crossed the wrong bits are not too bad
Or
Do legwork to figure out what is wrong defeating the purpose?
AND how do know your source is correct?#Ai continuing to learn will keep reintroducing bogusness exponentially!?
@Crystal_Fish_Caves @jenniferplusplus
This does remind me of this fucking weirdness when buying a house:
A lot of the US does not have the government keep track of who owns what land so when you buy a place, you need to also buy insurance that says that you are actually buying it from someone able to sell it.
As far as I can tell every other country just has a department that you can ask "hey is this the owner" and trust the answer.
-
@flippac @emily_s @jenniferplusplus Fiiiiine, there are also hardware errors; but doesn't that again come back to the human who designed the hardware?
@kerravonsen hey just to be clear, you're doing it right now. You're saying the computer is permitted to be wrong. The consequences will land on whoever was able to avoid them, and they will deserve it for not getting out of the way
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720I think that being liable for the mistakes an AI that you use is only fair... They who live by the sword etc.
-
I think that being liable for the mistakes an AI that you use is only fair... They who live by the sword etc.
@Daniel_Blake @jenniferplusplus The problems start if you aren't using the AI because you want to, but because you got ordered to use it.
Cory Doctorow has written a lot about what he calls Reverse Centaurs - persons having to work for a machine instead of persons using a machine. For instance:
https://pluralistic.net/2025/12/05/pop-that-bubble/#u-washington -
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720 -
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus i agree, but I also think that LLMs being unreliable is part of the business model, if it gave acceptable answers first time you'd only ask one question, if it messes up slightly you type more stuff, you rephrase the prompt or rewrite the spec, all of which are more tokens that your org will actually pay for. its like builtin #enshittification from the start.
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus AI appears to "learn from its mistakes" and amplify them...
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus
AI is a nifty tool, but blindly trusting its output is foolish. AI should not be treated as an unquestionable authority, which I've personally see happen in the workplace. The novelty of AI makes it enjoyable for now, yet companies rushing to replace human experience and expertise with AI will soon see quality erode and trust vanish altogether. When that happens these companies will learn that once quality and trust are lost, winning them back is far harder than maintaining them. -
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus ai is the intern on seven tabs of acid. He can no longer tell the difference between truth and fiction, and this will lead to lots of mistakes, most of which will lead to you staring at your monitor in confusion.
You must at a minimum verify the work, make sure it corresponds to reality, and get ready to wtf. -
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus Note well: whether “you” are actually liable for the errors made in the output produce by AI in response to your prompting depends entirely on whether “you” are someone privileged with impunity for your own errors in judgment or you instead are someone accountable for forced errors outside your own control.
-
@kerravonsen hey just to be clear, you're doing it right now. You're saying the computer is permitted to be wrong. The consequences will land on whoever was able to avoid them, and they will deserve it for not getting out of the way
@jenniferplusplus I am quite confused as to how you concluded that I said that, when I've been pointing out that it is human error
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720LLMs do not make mistakes on their own, you make mistakes using them
> "AI can make mistakes, always check the results"
> I fucking loathe this phrase and everything that goes into it.
Why? It is good advice and important when using LLMs.
I use LLMs every day in my coding practice, and they do make errors (thank you compiler)
LLMs are a tool, and must be wielded. When you use them you are responsible for the results
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus There's a misunderstanding, an "AI can" is like a "worms can", that's the subject. Now it all makes sense.
-
@emily_s @jenniferplusplus
As a computer programmer, yes. There is no such thing as a computer error. It is one or more of:
* programmer error
* documentation error
* user error (with a side-order of either documentation error or "user didn't bother to read the documentation")@kerravonsen @emily_s @jenniferplusplus or a gamma ray and bit flip. But that should probably be caught.
-
@kerravonsen hey just to be clear, you're doing it right now. You're saying the computer is permitted to be wrong. The consequences will land on whoever was able to avoid them, and they will deserve it for not getting out of the way
@jenniferplusplus The computer is wrongly permitted to be wrong. I thought I was agreeing with you.
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720AI *WILL* make mistakes. Do not use.
-
"AI can make mistakes, always check the results"
I fucking loathe this phrase and everything that goes into it. It's not advice. It's a threat.
You probably read it as "AI is _capable_ of making mistakes; you _should_ check the results".
What it actually says is "AI is _permitted_ to make mistakes; _you are liable_ for the results, whether you check them or not".
Except "you" is generally not even the person building, installing, or even using the AI. It's the person the AI is used on:
https://thepit.social/@peter/116205452673914720@jenniferplusplus
They want us to pay for a service they won't stand behind. That should tell you everything you need to know. -
@Crystal_Fish_Caves @jenniferplusplus
This does remind me of this fucking weirdness when buying a house:
A lot of the US does not have the government keep track of who owns what land so when you buy a place, you need to also buy insurance that says that you are actually buying it from someone able to sell it.
As far as I can tell every other country just has a department that you can ask "hey is this the owner" and trust the answer.
@gbargoud @Crystal_Fish_Caves @jenniferplusplus if the American insurance industry can find a way to require insurance for something, they will
