LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
Dare Obasanjo (@carnage4life@mas.to)
Attached: 1 image Hallucinations are the bane of my existence when using Claude Code and that has significantly improved after adding the following instructions to my Claude .MD file. Sharing for anyone else who uses Claude as a daily driver for analysis and writing but has gotten frustrated by it making things up.
mas.to (mas.to)
@benjamineskola I am a safety engineer for safety-relevant and mission-critical systems. And it is disheartening to see safety professionals at international conferences present 2-page-long prompts, doing basically all this, but much more so, and expect their "spicy autocomplete machine" (@pluralistic) to create safety analyses this way. And always talking about the LLM as if it could think. And prompt it to show its internal steps and "reasoning", not understanding that it does no such thing. It just creates another string of words that sounds as if an intelligence were describing the inner workings.
The upshot almost always is "It sucks, we have to check and correct everything. We love it. It is the future!"
🤪 -
@MissConstrue @complexmath @nelson @benjamineskola hypothetically it is possible for an artificial agent (read: "AI") to be capable of accurately modeling the world and "thinking", however it seems that this is not currently even remotely the case.
@solonovamax @MissConstrue @complexmath @nelson I would not expect a large language model to be capable of doing so, no matter how advanced. An ‘AI’ ‘agent’ based on some other technology? Perhaps. But at that point we’re literally just saying ‘technically it’s not impossible for this to exist in future’; we’re in the realm of science fiction.
-
@junkman I see your car analogy and I raise you a slavery analogy: that, too, had “proved some degree of usefulness” for “some people very good at wielding it as a tool”.
And I bet you’d rather not know how it worked (works) if you were the one finding it handy for the benefits it provided you. Saying you don’t partake while loudly proclaiming its usefulness is not fooling anyone.
@mushroom_man that analogy is also useful.
I don't like that the whole LLM functionality is based on stealing humanity's knowledge for profit. Outright violated (shitty) copyright law and basically the regular people got screwed over while mega corporations just agreed not to make a big fuzz about it.
The foundations are so rotten and just so people can chat with their computers instead of doing manual pointing and typing.
-
@SheRaPantsuit @Beatpoet13 Yeah, there’s probably something to be said about UI affordances or something like that, where the chat interface guides people into assuming intentionality where there is none, and where some other presentation might be more objectively received.
@benjamineskola @SheRaPantsuit @Beatpoet13
It's even worse. Interactive LLMs create a linguistic bypass channel that "connects" parts in our minds/brains that are ordinarily separated by filters for plausibility and attenuating uncontrolled feedback. Furthermore, they can be tailored to adversely amplify select thought patterns.
They're the first, rudimentary implementation of the kind of cognitohazards that used to be science fiction.
Already they're potent cult-indoctrination machines.
1/
-
@SheRaPantsuit @Beatpoet13 Yeah, there’s probably something to be said about UI affordances or something like that, where the chat interface guides people into assuming intentionality where there is none, and where some other presentation might be more objectively received.
@benjamineskola @SheRaPantsuit
hm not being near fluent in tech, kindly elaborate on what this means 'cause I do live on a curiosity>confusion dynamic ... -
@solonovamax @MissConstrue @complexmath @nelson I would not expect a large language model to be capable of doing so, no matter how advanced. An ‘AI’ ‘agent’ based on some other technology? Perhaps. But at that point we’re literally just saying ‘technically it’s not impossible for this to exist in future’; we’re in the realm of science fiction.
@benjamineskola @MissConstrue @complexmath @nelson I'm using the word "agent" to not necessarily refer to "AI agents"
see: https://tech.lgbt/@solonovamax/116659064720106166
but yes, I currently believe that an artificial agent capable of thought and accurately modeling the world is science fiction
however I believe it is possible, only based on the fact that the transformer architecture is turing complete. but it might not be efficient for this, it might require like a model that's 10,000x larger than what is currently the largest possible model. I do not believe it is something that is possible in the near future (well, I hope it isn't). -
@Aedius @benjamineskola language consists of two parts, the form and the sign
the form is the "tangible" part of the language, e.g. this text, or whatever soundwave physics bullshit is happening when we talk
the sign is the meaning, what you might visualize in your head when you read the word "cat"
LLMs only have access to form, so when the meaning of text is important (read: always), LLMs are not very useful
to this, promptfondlers always reply "but today is the worst it's ever gonna be"
@SheRaPantsuit @benjamineskola
I will add that to my presentation for the nexts teams at work ^^
-
@benjamineskola @MissConstrue @complexmath @nelson I'm using the word "agent" to not necessarily refer to "AI agents"
see: https://tech.lgbt/@solonovamax/116659064720106166
but yes, I currently believe that an artificial agent capable of thought and accurately modeling the world is science fiction
however I believe it is possible, only based on the fact that the transformer architecture is turing complete. but it might not be efficient for this, it might require like a model that's 10,000x larger than what is currently the largest possible model. I do not believe it is something that is possible in the near future (well, I hope it isn't).@benjamineskola @MissConstrue @complexmath @nelson the word "agent" is used to mean an actor performing actions to achieve their goal
it can be from something as simple as a thermometer that measures the inside temperature to adjust an HVAC system to remain within a given range (the actions being turning on/off the heat/cooling & the goal being to achieve a given temperature), to as complex as a human -
@benjamineskola @SheRaPantsuit
hm not being near fluent in tech, kindly elaborate on what this means 'cause I do live on a curiosity>confusion dynamic ...@Beatpoet13 @SheRaPantsuit what we’re saying is that they’re not lying; they can’t lie because they don’t think.
They’re just producing random text, basically, and sometimes that’s ‘true’ and sometimes it isn’t. But there’s no mind behind it.
(But the human brain tends to assume there’s intelligence present even when it’s just a mechanical process, and — I think — the way they’re presented magnifies that tendency.)
-
@MissConstrue @complexmath @nelson @benjamineskola hypothetically it is possible for an artificial agent (read: "AI") to be capable of accurately modeling the world and "thinking", however it seems that this is not currently even remotely the case.
@solonovamax @complexmath @nelson @benjamineskola
Theoretically, yes. But I point you towards one of my favorite long term AI projects, Cyc. (Whom I haven’t checked up on since Doug died.). I knew Doug since the 80s, and sort of stayed in the loop with what they were doing, because trying to develop an ontological system fascinates me. I want it to happen, I just don’t think we have the compute yet. I’m not sure it can be done with anything less than quantum.
But Doug’s vision of AI had nothing really in common with current LLM, especially since nobody has come up with a way for it to be commercially viable, because that wasn’t the vision. Knowledge was the dream. As electric dreams go, it was a pretty groovy one.
-
@benjamineskola @Beatpoet13 it can be helpful to replace an llm with "repeatedly pressing the suggested word on your phone keyboard."
If it spits out "I am a funny hamster", you wouldn't say it lied.
Humans are just not conditioned --- not wired, frankly --- to engage with machine generated, syntactically valid text. We suck at it. The ELIZA Effect wins every time.
@SheRaPantsuit @benjamineskola
I have disabled "suggestions" within seconds from getting a phone, it 's so symbolic for abusive interference, not only following into yr conversation but then pretending to either read yr mind or simply know best how to help ya out, if it be a person them 'd get urged to leave afaycr ... -
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
Dare Obasanjo (@carnage4life@mas.to)
Attached: 1 image Hallucinations are the bane of my existence when using Claude Code and that has significantly improved after adding the following instructions to my Claude .MD file. Sharing for anyone else who uses Claude as a daily driver for analysis and writing but has gotten frustrated by it making things up.
mas.to (mas.to)
@benjamineskola you're clearly not prompting it hard enough. i always add "DON'T MAKE MISTAKES" to my rules.md ...
these people are fucking beyond help, seriously...
-
@benjamineskola you're clearly not prompting it hard enough. i always add "DON'T MAKE MISTAKES" to my rules.md ...
these people are fucking beyond help, seriously...
@benjamineskola but it's ok, i have a second instance of a different LLM tasked with checking the output of the first LLM is CORRECT...
-
@benjamineskola @MissConstrue @complexmath @nelson I'm using the word "agent" to not necessarily refer to "AI agents"
see: https://tech.lgbt/@solonovamax/116659064720106166
but yes, I currently believe that an artificial agent capable of thought and accurately modeling the world is science fiction
however I believe it is possible, only based on the fact that the transformer architecture is turing complete. but it might not be efficient for this, it might require like a model that's 10,000x larger than what is currently the largest possible model. I do not believe it is something that is possible in the near future (well, I hope it isn't).@solonovamax yeah but that’s what I’m saying: if we’re discussing actually-existing “AI” agents it’s impossible and if you want to discuss hypothetical future AI it’s kind of unrelated.
-
@benjamineskola @SheRaPantsuit @Beatpoet13
It's even worse. Interactive LLMs create a linguistic bypass channel that "connects" parts in our minds/brains that are ordinarily separated by filters for plausibility and attenuating uncontrolled feedback. Furthermore, they can be tailored to adversely amplify select thought patterns.
They're the first, rudimentary implementation of the kind of cognitohazards that used to be science fiction.
Already they're potent cult-indoctrination machines.
1/
@benjamineskola @SheRaPantsuit @Beatpoet13
They not capable of this, because they're sophisticated, or in any way "intelligent". They're just creating bypass shorcuts in their user's brains/minds that ordinarily have filtering barriers inbetween. But without that, the user's brain/mind is feeding itself with its own – distorted – thought pattern, misattributing it for someone else's thoughts. Which is pretty much the same as what happens in the brains of people afflicted with schizophrenia.
2/ -
@SheRaPantsuit @benjamineskola
I have disabled "suggestions" within seconds from getting a phone, it 's so symbolic for abusive interference, not only following into yr conversation but then pretending to either read yr mind or simply know best how to help ya out, if it be a person them 'd get urged to leave afaycr ...@Beatpoet13 yes, I also really dislike those ‘autocomplete’ things when writing, it’s quite horrible. I’ll say what I choose, not what the machine thinks is probable.
-
@solonovamax @complexmath @nelson @benjamineskola
Theoretically, yes. But I point you towards one of my favorite long term AI projects, Cyc. (Whom I haven’t checked up on since Doug died.). I knew Doug since the 80s, and sort of stayed in the loop with what they were doing, because trying to develop an ontological system fascinates me. I want it to happen, I just don’t think we have the compute yet. I’m not sure it can be done with anything less than quantum.
But Doug’s vision of AI had nothing really in common with current LLM, especially since nobody has come up with a way for it to be commercially viable, because that wasn’t the vision. Knowledge was the dream. As electric dreams go, it was a pretty groovy one.
@MissConstrue @complexmath @nelson @benjamineskola I don't think you necessarily need anything quantum for it, just some very powerful classical computational device (based on the fact that given our current understanding of the human brain, they do not do any quantum-like processing. also the fact that quantum computers have been "just a few years away" for what, like 30 years now?)
but yeah, I find AI in concept quite interesting. if you'd asked me pre-chatgpt, that's what I wanted to specialize into.
now? god, that shit just sounds so fucking exhausting. I do not want anything to do with it. I want to stay as far away from it as possible. -
@Beatpoet13 in all seriousness I don’t like the terminology of ‘lying’ here either. It implies intent.
It’s not a lie for the same reason that it’s not a hallucination; there’s no difference from the LLM’s perspective. It’s not capable of evaluating the truth-value of its output, much less intentionally producing untrue (or true) statements. It’s mere probability.
Responsible usage of these tools would involve mechanisms to increase the probability of the desired output, but pretending it’s capable of evaluating that itself will not help at all.
@benjamineskola seems my cynicism is taken quite seriously, obviously a non entity has no capacity to lie since that would imply actual personified existence, not a datafiltering mechanism rife with programmer bias, misconception & projected as a digital saviour rehash, after paying dearly to peek into how business"schools" dish out "Ai implementation training" it became quite clear that this is a whitewashing of empirical new clothes unlike any bubble yet, said one Ai to the other, does it hurt
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
Dare Obasanjo (@carnage4life@mas.to)
Attached: 1 image Hallucinations are the bane of my existence when using Claude Code and that has significantly improved after adding the following instructions to my Claude .MD file. Sharing for anyone else who uses Claude as a daily driver for analysis and writing but has gotten frustrated by it making things up.
mas.to (mas.to)
@benjamineskola Telling an LLM not to make stuff up isn't even like telling your dog not to pull on a leash ... because you can actually train a dog not to pull on a leash.
But squirrels or that cat down the street or a random stranger that looks like they might have treats will though off even a well trained dog
This is a post about the futility of better training for LLMs.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
Dare Obasanjo (@carnage4life@mas.to)
Attached: 1 image Hallucinations are the bane of my existence when using Claude Code and that has significantly improved after adding the following instructions to my Claude .MD file. Sharing for anyone else who uses Claude as a daily driver for analysis and writing but has gotten frustrated by it making things up.
mas.to (mas.to)
This is why coding is the most "successful" (please note the quote marks, I think coding with these things is absolutely a net negative) use case.
Coding operates in a much more limited space on much more clearly defined terms than human languages.
