File under: AI Resistance
-
File under: AI Resistance
I'm reading – and enjoying immensely – @emilymbender and @alex's book #TheAICon.
I've been learning a lot of things and something that really stood out recently is the need to be careful about the language we use to describe these systems. Bender and Hanna helpfully explain:
"It matters what words we use when we talk about these technologies. For instance, in our writing, we don’t use the term “hallucination” to discuss the errors of LLMs, for two reasons. First, if it’s used tongue-in-cheek, it is making light of what can be symptoms of serious mental illness. Second,
“hallucination” refers to the experience of perceiving things that aren’t there. But LLMs actually don’t have perceptions, and suggesting that they do is yet more unhelpful anthropomorphization. That means we also avoid assigning thought processes to these systems, or saying that they can
“think”. Metaphors have power, they structure the frames of discourse, and they can subtly and insidiously encourage certain ways of understanding technology and the social systems it is embedded in."Antropomorphizing AI contributes to AI hype. Thanks Emily and Alex for helping me see things this way!
-
File under: AI Resistance
I'm reading – and enjoying immensely – @emilymbender and @alex's book #TheAICon.
I've been learning a lot of things and something that really stood out recently is the need to be careful about the language we use to describe these systems. Bender and Hanna helpfully explain:
"It matters what words we use when we talk about these technologies. For instance, in our writing, we don’t use the term “hallucination” to discuss the errors of LLMs, for two reasons. First, if it’s used tongue-in-cheek, it is making light of what can be symptoms of serious mental illness. Second,
“hallucination” refers to the experience of perceiving things that aren’t there. But LLMs actually don’t have perceptions, and suggesting that they do is yet more unhelpful anthropomorphization. That means we also avoid assigning thought processes to these systems, or saying that they can
“think”. Metaphors have power, they structure the frames of discourse, and they can subtly and insidiously encourage certain ways of understanding technology and the social systems it is embedded in."Antropomorphizing AI contributes to AI hype. Thanks Emily and Alex for helping me see things this way!
@elena @emilymbender @alex do they suggest an alternative term to use?
-
@elena @emilymbender @alex do they suggest an alternative term to use?
@rpbook I obviously can't speak for @emilymbender @alex but personally I try to use terms like "model creation" in lieu of "training"; "output" instead of "answer" or "response"; etc. Also pointing out that so-called "hallucinations" in generative AI is simply how they are designed to work. See https://michael.kjorling.se/ai/ for how this can work out in practice.
-
R relay@relay.mycrowd.ca shared this topic
-
@rpbook I obviously can't speak for @emilymbender @alex but personally I try to use terms like "model creation" in lieu of "training"; "output" instead of "answer" or "response"; etc. Also pointing out that so-called "hallucinations" in generative AI is simply how they are designed to work. See https://michael.kjorling.se/ai/ for how this can work out in practice.
@mkj @rpbook @emilymbender @alex @elena I'm also interested in what alternative terminology should be used... "error prone output" seems appropriate but perhaps a bit long?
-
@mkj @rpbook @emilymbender @alex @elena I'm also interested in what alternative terminology should be used... "error prone output" seems appropriate but perhaps a bit long?
@eobet I think it depends a lot on the context: what actually even fits? I'll give e.g. "hallucinate" that; it's easy to work in. Sometimes I suppose simply "incorrect" fits the bill, especially if the aim is to describe a property of the output.
IMO not everything necessarily requires a single word.
-
R relay@relay.an.exchange shared this topic
-
File under: AI Resistance
I'm reading – and enjoying immensely – @emilymbender and @alex's book #TheAICon.
I've been learning a lot of things and something that really stood out recently is the need to be careful about the language we use to describe these systems. Bender and Hanna helpfully explain:
"It matters what words we use when we talk about these technologies. For instance, in our writing, we don’t use the term “hallucination” to discuss the errors of LLMs, for two reasons. First, if it’s used tongue-in-cheek, it is making light of what can be symptoms of serious mental illness. Second,
“hallucination” refers to the experience of perceiving things that aren’t there. But LLMs actually don’t have perceptions, and suggesting that they do is yet more unhelpful anthropomorphization. That means we also avoid assigning thought processes to these systems, or saying that they can
“think”. Metaphors have power, they structure the frames of discourse, and they can subtly and insidiously encourage certain ways of understanding technology and the social systems it is embedded in."Antropomorphizing AI contributes to AI hype. Thanks Emily and Alex for helping me see things this way!
@elena @emilymbender @alex Thanks for that review and excerpt!
The words used for this technology has bugged me since it became a thing. Starting with the fact that it is not AI, it's not intelligence. I hadn't even gotten into the other words being used, and I have a lot of respect that they are digging into that.
I may have to pick up this book, and I don't read much non-fiction.
-
R relay@relay.infosec.exchange shared this topic
