As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.
-
@wolf480pl @leeloo The OP is saying that it literally lacks the capacity for original thought - it is a parrot, repeating sounds without understanding of the concepts behind them.
It's not like a termite, whose mound creation behavior can be replicated by a simple ruleset but that exists as a fully functional living organism in the context of a complex environment where choices must be grounded in the shared physical world for the organism to survive.
It's not about how the neurons are arranged. It's about what kinds of representation they're capable of and what kinds of functions they can perform.
We've created a funhouse mirror that's reflecting us in unprecedented detail and has been finetuned to reflect what we do when we express selfhood.
@wolf480pl @leeloo
Melissa Scott wrote a beautiful pair of novels about this: Dreamships and Dreaming Metal.In Dreamships, an AI has been programmed to think it is sentient and starts killing people. If it has an accurate model of the person, killing the person doesn't matter, because the person *is* the model and it has a copy of them. It literally cannot see the difference because creating the concept of there being a difference would violate its core programming that its own model counts as a living being.
In Dreaming Metal, an AI operating metal bodies as part of a magic act is given a musical instrument with an electronic interface. Its grounding in the physical world, with human performers, enables it to develop a sense of self and choose its own path as a musician.
These are fiction, but it's the best, most accessible illustration of the difference between funhouse mirror stochastic parrots and sentient agents that I've run across.
Dreamships
Read 45 reviews from the world’s largest community for readers. Dreamships is the story of a freelance space pilot and her crew, who are hired by a rich co…
Goodreads (www.goodreads.com)
-
As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.
It's literally a description of how they work.
The so-called training data is used to build a huge database of words and the probability of them fitting together.
Stochastic because the whole thing is statistics.
Parrot because the answer is just repeating the most probable word combinations from its training dataset.Calling an LLM a stochastic parrot is lile calling a car a motorised vehicle with wheels. It doesn't say anything about cars being good or bad. It does, however, take away the magic. So if you feel a need to defend AI when you hear the term stochastic parrot, consider that you may have elevated them to a god-like status, and that's why you go on the defense when the magic is dispelled.
I think stochastic parrot is one of the kinder things that can be said.
-
@leeloo The thing is, how can we sure that human intelligence does not essentially work in the same way? My Christian believe tells me we have a soul and LLM's do not, that may be the difference. But from an agnostic perspective, we might reach the point where one cannot tell the difference.
@tobifant @leeloo Whilst we obviously can't show if humans have a soul, we can absolutely show that humans have e.g. abstracted concept frameworks that are not solely based on averages of language statistics. I understand what an "owl" is, for example, in a way separate to the numerical relationships between the word "owl" and other words. That is a really fundamental information processing difference and allows me to construct *novel* understandings of that concept in ways that an LLM couldn't.
-
@wolf480pl @leeloo
Melissa Scott wrote a beautiful pair of novels about this: Dreamships and Dreaming Metal.In Dreamships, an AI has been programmed to think it is sentient and starts killing people. If it has an accurate model of the person, killing the person doesn't matter, because the person *is* the model and it has a copy of them. It literally cannot see the difference because creating the concept of there being a difference would violate its core programming that its own model counts as a living being.
In Dreaming Metal, an AI operating metal bodies as part of a magic act is given a musical instrument with an electronic interface. Its grounding in the physical world, with human performers, enables it to develop a sense of self and choose its own path as a musician.
These are fiction, but it's the best, most accessible illustration of the difference between funhouse mirror stochastic parrots and sentient agents that I've run across.
Dreamships
Read 45 reviews from the world’s largest community for readers. Dreamships is the story of a freelance space pilot and her crew, who are hired by a rich co…
Goodreads (www.goodreads.com)
@robotistry
@leeloo
so it's a parrot not because it's a matrix of probabilities, but because its hasn't experienced the real-world consequences of its words/actions and updated the probabilities based on those consequences? -
@leeloo I hadn't thought about it as being something that takes magic away from folks like that. Honestly I always found it an accurate shortcut term for what's genuinely a fascinating but hilariously misused technology.
I think the worst part is then when folks hear "statistics" and go "See this is why it's safe to feed it raw data" and it's like oh my god NO.
-
-
@lmorchard The ability to induce such a rule goes well beyond the OP's characterisation of what LLMs do.
@mudri @lmorchard it’s not inductive at all though. It’s just parroting the patterns it sees in its training data. If it wasn’t common to see exchanges like that, the response would be utter nonsense.
People misunderstand what “training” is. It’s modeling the input. Humans develop the rules for how to model that input. Emergent properties of that process can easily *seem* like thinking or reason, but it’s an illusion.
-
@leeloo I feel like there are certain situations where a stochastic parrot is useful, many more situations where it is not, and alarmingly few people recognizing the difference.
-
@leeloo on the flipside, I feel like some people use the term "stochastic parrot" or "it just completes the next token" to imply that "therefore it cannot be intelligent" - is that correct reasoning?
Yes and I take that position.
-
@leeloo @wolf480pl @lmorchard I mean, I believe the human mind is the product of the physical human, largely of the brain (I don't believe in a non-physical soul), and it might indeed be basically an incredibly complex big bunch of matrix multiplications. And yeah I believe that's pretty magical.
@dragonfrog @leeloo @wolf480pl
"Imagine you have two machines. One you can open up and examine all of its workings, and if you give it every picture of a cat on the whole internet, it can reliably distinguish cats from non-cats. The other is a black box and it can also reliably distinguish cats from non-cats if you give it half a dozen pictures of cats, some apple sauce, and a hug. ... I am extremely confident in saying it doesn’t work the same way as the first one."
A.I. Isn't People
How many Reddit posts does it take to learn to read?
Today in Tabs (www.todayintabs.com)
-
@lmorchard @leeloo
I don't buy a general "no matrix multiplication will ever be intelligent".@wolf480pl @lmorchard @leeloo you are allowed to believe that even if it is factually incorrect.
A non-anthropomorphized view of LLMs
In many discussions where questions of "alignment" or "AI safety" crop up, I am baffled by seriously intelligent people imbuing almost magic...
(addxorrol.blogspot.com)
Is language the same as intelligence? The AI industry desperately needs it to be
Neuroscience indicates language is distinct from thought, raising questions about whether AI large language models are a viable path to artificial general intelligence.
The Verge (www.theverge.com)
The LLMentalist Effect: how chat-based Large Language Models rep…
How to make better software with systems-thinking
Out of the Software Crisis (softwarecrisis.dev)
-
@leeloo
My point is that emergent properties can manifest even in systems ruled by very simple rules, and can be difficult to predict by just looking at the rules.And human intelligence, whatever it is, is likely an emergent property of human brain.
Therefore, we cannot rule out that a similar emergent property will appear in artidicial systems that are not made of neurons without referring to how the neurons are arranged, and how the artificial systems are arranged.
@wolf480pl @leeloo dude its a spreadsheet
-
@robotistry
@leeloo
so it's a parrot not because it's a matrix of probabilities, but because its hasn't experienced the real-world consequences of its words/actions and updated the probabilities based on those consequences?@wolf480pl @robotistry @leeloo spreadsheets cant have experiences, it doesnt update its probabilities, human beings spend insane amounts of money to generate the spreadsheets, nothing new comes out of them, have you uhh ever like looked into how this software works?
-
@lmorchard @leeloo
I don't buy a general "no matrix multiplication will ever be intelligent".@wolf480pl @lmorchard @leeloo praise be all glory to the llm
-
@lmorchard @leeloo
I don't buy a general "no matrix multiplication will ever be intelligent".@wolf480pl @lmorchard @leeloo okay but that’s true. matrix multiplication will never be intelligent. the truth is neat!
-
@leeloo the flip side question about intelligence and LLMs is whether much of what we consider intelligence in humans is in fact just stochastic parrotting by humans.
@clusterfcku @leeloo it’s not, and it sucks to suggest that
-
@wolf480pl @lmorchard @leeloo praise be all glory to the llm
-
@wolf480pl @lmorchard @leeloo you are allowed to believe that even if it is factually incorrect.
A non-anthropomorphized view of LLMs
In many discussions where questions of "alignment" or "AI safety" crop up, I am baffled by seriously intelligent people imbuing almost magic...
(addxorrol.blogspot.com)
Is language the same as intelligence? The AI industry desperately needs it to be
Neuroscience indicates language is distinct from thought, raising questions about whether AI large language models are a viable path to artificial general intelligence.
The Verge (www.theverge.com)
The LLMentalist Effect: how chat-based Large Language Models rep…
How to make better software with systems-thinking
Out of the Software Crisis (softwarecrisis.dev)
@jrdepriest @wolf480pl @leeloo I'm confused... those links basically say what I said. (i.e. the "intelligence" is second-hand) That's... incorrect?
-
@jrdepriest @wolf480pl @leeloo I'm confused... those links basically say what I said. (i.e. the "intelligence" is second-hand) That's... incorrect?
LLM based genAI can never be "intelligent". They can spit out language that looks like intelligence but there is no thinking, no inner life, no thoughts, just math. And this is not how the human brain works.
Also, we know the brain is not a computer.
-
@dragonfrog @leeloo @wolf480pl
"Imagine you have two machines. One you can open up and examine all of its workings, and if you give it every picture of a cat on the whole internet, it can reliably distinguish cats from non-cats. The other is a black box and it can also reliably distinguish cats from non-cats if you give it half a dozen pictures of cats, some apple sauce, and a hug. ... I am extremely confident in saying it doesn’t work the same way as the first one."
A.I. Isn't People
How many Reddit posts does it take to learn to read?
Today in Tabs (www.todayintabs.com)
@lmorchard @leeloo @wolf480pl good grief now I have to sound like Sam friggin Altman, and there is clearly something very wrong with that man.
But your description ignores that humans need a solid 6 months of "training data" to get object permanence, never mind the concept of categories or species of animals, never mind understanding the category differences between cats and foxes well enough to reliably tell one from the other.
