As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.
-
-
@lmorchard The ability to induce such a rule goes well beyond the OP's characterisation of what LLMs do.
@mudri @lmorchard it’s not inductive at all though. It’s just parroting the patterns it sees in its training data. If it wasn’t common to see exchanges like that, the response would be utter nonsense.
People misunderstand what “training” is. It’s modeling the input. Humans develop the rules for how to model that input. Emergent properties of that process can easily *seem* like thinking or reason, but it’s an illusion.
-
@leeloo I feel like there are certain situations where a stochastic parrot is useful, many more situations where it is not, and alarmingly few people recognizing the difference.
-
@leeloo on the flipside, I feel like some people use the term "stochastic parrot" or "it just completes the next token" to imply that "therefore it cannot be intelligent" - is that correct reasoning?
Yes and I take that position.
-
@leeloo @wolf480pl @lmorchard I mean, I believe the human mind is the product of the physical human, largely of the brain (I don't believe in a non-physical soul), and it might indeed be basically an incredibly complex big bunch of matrix multiplications. And yeah I believe that's pretty magical.
@dragonfrog @leeloo @wolf480pl
"Imagine you have two machines. One you can open up and examine all of its workings, and if you give it every picture of a cat on the whole internet, it can reliably distinguish cats from non-cats. The other is a black box and it can also reliably distinguish cats from non-cats if you give it half a dozen pictures of cats, some apple sauce, and a hug. ... I am extremely confident in saying it doesn’t work the same way as the first one."
A.I. Isn't People
How many Reddit posts does it take to learn to read?
Today in Tabs (www.todayintabs.com)
-
@lmorchard @leeloo
I don't buy a general "no matrix multiplication will ever be intelligent".@wolf480pl @lmorchard @leeloo you are allowed to believe that even if it is factually incorrect.
A non-anthropomorphized view of LLMs
In many discussions where questions of "alignment" or "AI safety" crop up, I am baffled by seriously intelligent people imbuing almost magic...
(addxorrol.blogspot.com)
Is language the same as intelligence? The AI industry desperately needs it to be
Neuroscience indicates language is distinct from thought, raising questions about whether AI large language models are a viable path to artificial general intelligence.
The Verge (www.theverge.com)
The LLMentalist Effect: how chat-based Large Language Models rep…
How to make better software with systems-thinking
Out of the Software Crisis (softwarecrisis.dev)
-
@leeloo
My point is that emergent properties can manifest even in systems ruled by very simple rules, and can be difficult to predict by just looking at the rules.And human intelligence, whatever it is, is likely an emergent property of human brain.
Therefore, we cannot rule out that a similar emergent property will appear in artidicial systems that are not made of neurons without referring to how the neurons are arranged, and how the artificial systems are arranged.
@wolf480pl @leeloo dude its a spreadsheet
-
@robotistry
@leeloo
so it's a parrot not because it's a matrix of probabilities, but because its hasn't experienced the real-world consequences of its words/actions and updated the probabilities based on those consequences?@wolf480pl @robotistry @leeloo spreadsheets cant have experiences, it doesnt update its probabilities, human beings spend insane amounts of money to generate the spreadsheets, nothing new comes out of them, have you uhh ever like looked into how this software works?
-
@lmorchard @leeloo
I don't buy a general "no matrix multiplication will ever be intelligent".@wolf480pl @lmorchard @leeloo praise be all glory to the llm
-
@lmorchard @leeloo
I don't buy a general "no matrix multiplication will ever be intelligent".@wolf480pl @lmorchard @leeloo okay but that’s true. matrix multiplication will never be intelligent. the truth is neat!
-
@leeloo the flip side question about intelligence and LLMs is whether much of what we consider intelligence in humans is in fact just stochastic parrotting by humans.
@clusterfcku @leeloo it’s not, and it sucks to suggest that
-
@wolf480pl @lmorchard @leeloo praise be all glory to the llm
-
@wolf480pl @lmorchard @leeloo you are allowed to believe that even if it is factually incorrect.
A non-anthropomorphized view of LLMs
In many discussions where questions of "alignment" or "AI safety" crop up, I am baffled by seriously intelligent people imbuing almost magic...
(addxorrol.blogspot.com)
Is language the same as intelligence? The AI industry desperately needs it to be
Neuroscience indicates language is distinct from thought, raising questions about whether AI large language models are a viable path to artificial general intelligence.
The Verge (www.theverge.com)
The LLMentalist Effect: how chat-based Large Language Models rep…
How to make better software with systems-thinking
Out of the Software Crisis (softwarecrisis.dev)
@jrdepriest @wolf480pl @leeloo I'm confused... those links basically say what I said. (i.e. the "intelligence" is second-hand) That's... incorrect?
-
@jrdepriest @wolf480pl @leeloo I'm confused... those links basically say what I said. (i.e. the "intelligence" is second-hand) That's... incorrect?
LLM based genAI can never be "intelligent". They can spit out language that looks like intelligence but there is no thinking, no inner life, no thoughts, just math. And this is not how the human brain works.
Also, we know the brain is not a computer.
-
@dragonfrog @leeloo @wolf480pl
"Imagine you have two machines. One you can open up and examine all of its workings, and if you give it every picture of a cat on the whole internet, it can reliably distinguish cats from non-cats. The other is a black box and it can also reliably distinguish cats from non-cats if you give it half a dozen pictures of cats, some apple sauce, and a hug. ... I am extremely confident in saying it doesn’t work the same way as the first one."
A.I. Isn't People
How many Reddit posts does it take to learn to read?
Today in Tabs (www.todayintabs.com)
@lmorchard @leeloo @wolf480pl good grief now I have to sound like Sam friggin Altman, and there is clearly something very wrong with that man.
But your description ignores that humans need a solid 6 months of "training data" to get object permanence, never mind the concept of categories or species of animals, never mind understanding the category differences between cats and foxes well enough to reliably tell one from the other.
-
As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.
It's literally a description of how they work.
The so-called training data is used to build a huge database of words and the probability of them fitting together.
Stochastic because the whole thing is statistics.
Parrot because the answer is just repeating the most probable word combinations from its training dataset.Calling an LLM a stochastic parrot is lile calling a car a motorised vehicle with wheels. It doesn't say anything about cars being good or bad. It does, however, take away the magic. So if you feel a need to defend AI when you hear the term stochastic parrot, consider that you may have elevated them to a god-like status, and that's why you go on the defense when the magic is dispelled.
-
@lmorchard @leeloo @wolf480pl good grief now I have to sound like Sam friggin Altman, and there is clearly something very wrong with that man.
But your description ignores that humans need a solid 6 months of "training data" to get object permanence, never mind the concept of categories or species of animals, never mind understanding the category differences between cats and foxes well enough to reliably tell one from the other.
@lmorchard @leeloo @wolf480pl I guess part of it is maybe that I don't think intelligence is some exclusively human thing. LLMs clearly aren't human-like intelligent. I'm personally confident they're not as intelligent as any primate.
But are they as intelligent as a shrimp? I think they've got to be more intelligent than a mosquito.
I wouldn't turn to a shrimp for advice but they're not *without* intelligence.
-
@lmorchard @leeloo @wolf480pl I guess part of it is maybe that I don't think intelligence is some exclusively human thing. LLMs clearly aren't human-like intelligent. I'm personally confident they're not as intelligent as any primate.
But are they as intelligent as a shrimp? I think they've got to be more intelligent than a mosquito.
I wouldn't turn to a shrimp for advice but they're not *without* intelligence.
@dragonfrog @lmorchard @leeloo @wolf480pl
Are the images reflected in a distorted mirror the product of intelligence (of the mirror)?
They are coherent, a literal transform of the input images, reflected and produce a recognizable, if distorted and changed version.
A traditional function output. Let's add some noise to make it non-deterministic, a wind blowing through that minutely distorts the surface.
Intelligible output following from the input, but the mirror itself isn't intelligent.
-
@dragonfrog @lmorchard @leeloo @wolf480pl
Are the images reflected in a distorted mirror the product of intelligence (of the mirror)?
They are coherent, a literal transform of the input images, reflected and produce a recognizable, if distorted and changed version.
A traditional function output. Let's add some noise to make it non-deterministic, a wind blowing through that minutely distorts the surface.
Intelligible output following from the input, but the mirror itself isn't intelligent.
@dragonfrog @lmorchard @leeloo @wolf480pl
The intelligence apparently making the meaning is pre-encoded in the input. Likewise, the vector math is extracting and exposing structure, encoded in language, put there originally by the intelligent humans.
There is no world model or understanding. That's why counting the "r" in strawberry or simply counting to 200 is so challenging.
The behavior can reasonably be called intelligent, but it's due to borrowed, reformulated, extracted intelligence
-
As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.
It's literally a description of how they work.
The so-called training data is used to build a huge database of words and the probability of them fitting together.
Stochastic because the whole thing is statistics.
Parrot because the answer is just repeating the most probable word combinations from its training dataset.Calling an LLM a stochastic parrot is lile calling a car a motorised vehicle with wheels. It doesn't say anything about cars being good or bad. It does, however, take away the magic. So if you feel a need to defend AI when you hear the term stochastic parrot, consider that you may have elevated them to a god-like status, and that's why you go on the defense when the magic is dispelled.
@leeloo As a side note, I sometimes worry about how much parroting happens in academia among humans even before/without LLMs, where people repeat things without understanding what they’re talking about. I guess at least for students, it sometimes is about learning to talk the talk, and then gradually developing more understanding and genuine thinking around topics. At least we humans are capable of developing that understanding if we bother to try.
