The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.
-
@riley That actually really clears up how I feel when I very occasionally test an LLM. It gives me an answer but I just cannot trust that answer unless I already know.
@edbo @riley This is also an illustration of why LLMs have a (very limited) utility in generating computer code.
Computer code has a specific purpose. The generated code can be tested against the task. This can be useful.
But computer code will also have other effects and costs that only a human can validate well.
At most an LLM should be used to generate rough drafts of well defined functions that will be reviewed and tuned by a qualified human.
-
The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.
A clock that always shows the same time is never right, even in the moments of the day when the time happens to be what it shows, because you don't gain any information about what time it is by looking at the clock.
This reasoning also applies to chatbots. If you can't tell whether what you have been given is useful information unless you alreay know the information, then you haven't been given useful information.
FWIW? There is a branch of philosophy focused on the problem you describe – one so old we use an ancient Greek name for it:
> https://en.wikipedia.org/wiki/Epistemology
This is because determining if information is true and actionable has *always* been fraught. AI merely adds a brand new way to get wrong information.
The underlying problem arises when people uncritically believe *anything* from *any source*; human or machine. This is why science has protocols for publishing and re-creating results.
-
@riley Let's say I constructed an elevator with 12 floors. The elevator stops at the next floor every hour on the hour starting from the ground floor at noon and returning to the ground floor at midnight at which point the process repeats. There is a window on the door which shows a broken clock for each floor. Ground floor clock is broken at 12, the next at 1 and so on.
Consider the nature of a fool who gets locked in the elevator and does not know the time. Does the broken clock inform him?
@Smohc_Stahc If we made a hammer out of dynamite, would it be a hammer or dynamite?
-
The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.
A clock that always shows the same time is never right, even in the moments of the day when the time happens to be what it shows, because you don't gain any information about what time it is by looking at the clock.
This reasoning also applies to chatbots. If you can't tell whether what you have been given is useful information unless you alreay know the information, then you haven't been given useful information.
@riley @matt But information always has a probability value attached to it. For the broken clock, it is pretty much 0% likely that the time will be correct (1 in 12 times 60 = 1 in 720). But for the LLM, the probability could be 70% to 90% depending on what kind of information you are asking it for and how good the specific LLM is. Information becomes more useful as the probability of it being correct approaches 100%. A good reliable source would have a much higher probability of being correct and therefore be more useful, but the LLM is closer to that than to a broken clock at least for most things.
-
@MissConstrue Are you a chatbot sycophanting me up?
These days, one can never be too cautious.
@riley Thats a very good question and you are so clever to think of it, I’d be happy to drill down on this topic for you.
Heh, sorry. Not a chatbot. Old philosopher, so...like a chatbot, only caffeine powered, argumentative and capable of consciousness. (Or at least, I would argue I’m conscious.) I honestly did believe it was a very illustrative analogy. Most people will parrot the clock paradigm; ie right twice a day, when you are correct that the underlying logic of the premise is faulty, and therefore any attempt to treat it as true will fail.
-
@riley @cptbutton I never really knew my root...
-
@riley Thats a very good question and you are so clever to think of it, I’d be happy to drill down on this topic for you.
Heh, sorry. Not a chatbot. Old philosopher, so...like a chatbot, only caffeine powered, argumentative and capable of consciousness. (Or at least, I would argue I’m conscious.) I honestly did believe it was a very illustrative analogy. Most people will parrot the clock paradigm; ie right twice a day, when you are correct that the underlying logic of the premise is faulty, and therefore any attempt to treat it as true will fail.
@MissConstrue There's an interesting pattern to a large number of these faults, but I guess it'll be a topic for another day.
-
The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.
A clock that always shows the same time is never right, even in the moments of the day when the time happens to be what it shows, because you don't gain any information about what time it is by looking at the clock.
This reasoning also applies to chatbots. If you can't tell whether what you have been given is useful information unless you alreay know the information, then you haven't been given useful information.
@riley Riley, are you aware that linguistics in the 60s established language use conveys meaning by reference to other language with no guaranteed relation with some external reality? So all words bear the same relationship with reality a stopped clock has with actual time.
I mention this because LLMs are not designed to provide information about the world, they're designed to generate discourse — language use (its output) that is validly constructed by reference to other language use (its training dataset). It's not fair to judge an LLM on the basis it's a lousy search engine.
But if you spin up a RAG like NotebookLM and give it a reality to refer to (a set of documents) and then ask it a question i.e. is XYZ in the document set, turns out LLMs can do a pretty good job of accurately answering yes or no.
-
@riley @matt But information always has a probability value attached to it. For the broken clock, it is pretty much 0% likely that the time will be correct (1 in 12 times 60 = 1 in 720). But for the LLM, the probability could be 70% to 90% depending on what kind of information you are asking it for and how good the specific LLM is. Information becomes more useful as the probability of it being correct approaches 100%. A good reliable source would have a much higher probability of being correct and therefore be more useful, but the LLM is closer to that than to a broken clock at least for most things.
@emassey0135 So it is with other commercial products. That's why there's rules specifying that berries for human consumption can't contain more than something like four aphids per a hundred grammes.
But who would buy jam with 30% aphid content? Even 10% aphid content, really?
-
@MissConstrue There's an interesting pattern to a large number of these faults, but I guess it'll be a topic for another day.
I was thinking of some equipment I saw at a "Telekom-Museum" in Germany - it contained a clock but wasn't always powered on (or was just a display piece)
The Germans had quite sensibly put a diagonal strip of red tape (in the style of the "Universal No" symbol) across the clock face, so you knew it was *not* a timepiece to be trusted..
-
I was thinking of some equipment I saw at a "Telekom-Museum" in Germany - it contained a clock but wasn't always powered on (or was just a display piece)
The Germans had quite sensibly put a diagonal strip of red tape (in the style of the "Universal No" symbol) across the clock face, so you knew it was *not* a timepiece to be trusted..
@vfrmedia In aviation, the process is standardised by way of the INOP stickers.
-
D drajt@fosstodon.org shared this topic
-
@riley I am sorry, this is not correct analogy
The bot not giving you correct information 100% of the time doesn't make them useless
A Search engine doesn't give you the correct answer all the time.
Chatbots are incredibly helpful. Don't take the answer as 100% correct, review and research accuracy after you get the answer but they save you immense amount of time from searching yourself
Think of them as hiring a jr employee or assistant. They are helpful but you must review their work -
@proedie @riley after obsessing a little over getting to the bottom of this, the answer seems to be that the historical origin (from 1711) is akin to "If you stop chasing trends you will sometimes be fashionable", which is more in line with riley's definition in the OP. The other "official" definitions I've found seem to follow this as well.
The definition that "coincidental correctness is worthless" seems to be a personal (though common) interpretation.
-
@Smohc_Stahc If we made a hammer out of dynamite, would it be a hammer or dynamite?
@riley This process turns dynamite into dynamite. The part is the whole.
However, the elevator is not the whole of the machine. It can be determined that the elevator tells time but which time is a mystery without the broken clocks. The elevator does not fix the clocks either, they are still broken.
-
@riley @MissConstrue I am not a bot. Please don't look at my name.
@bdf2121cc3334b35b6ecda66e471 @riley
01001001 00100000 01110011 01100101 01100101 00100000 01111001 01101111 01110101
-
@riley Riley, are you aware that linguistics in the 60s established language use conveys meaning by reference to other language with no guaranteed relation with some external reality? So all words bear the same relationship with reality a stopped clock has with actual time.
I mention this because LLMs are not designed to provide information about the world, they're designed to generate discourse — language use (its output) that is validly constructed by reference to other language use (its training dataset). It's not fair to judge an LLM on the basis it's a lousy search engine.
But if you spin up a RAG like NotebookLM and give it a reality to refer to (a set of documents) and then ask it a question i.e. is XYZ in the document set, turns out LLMs can do a pretty good job of accurately answering yes or no.
@onekind @riley The answer would still be fuzzy -- there would be a ratio of certainty associated to yes and no. Other methods like pattern search could be tuned to be completely certain on the yes or the no -- some even both -- but I think it is impossible to tune stochastic methods in the same way. To conclude, external data is needed to assess the correctness of the answer of an LLM.
-
@hypolite @riley
A computer is not a human, but tools can replace humans to do certain job if not better.
If you don't like dishwashers, laundry machines, sewing machines, tractors and diggers then by all means hire someone to do it, but most of us find it more effective to use machines instead
I would rather focus my time on building more complex things than waste it on doing less complex jobs that a machine (or AI) can easily do in less time -
@onekind @riley The answer would still be fuzzy -- there would be a ratio of certainty associated to yes and no. Other methods like pattern search could be tuned to be completely certain on the yes or the no -- some even both -- but I think it is impossible to tune stochastic methods in the same way. To conclude, external data is needed to assess the correctness of the answer of an LLM.
@pedromj @riley First, you're assuming that a RAG functions the same way as an LLM. It uses a mix of stochastic and deterministic analysis.
Second, a yes or no answer from a human is also 'fuzzy' in the sense that describing a query in language is never entirely precise, for exactly the reasons I discussed in my previous toot, so the answer given is always 'this is my best guess based on my contingent understanding of your imperfectly phrased question.'
Re your conclusion, I already described the document set as an artificially constructed external reality, which satisfies your objection.
-
The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.
A clock that always shows the same time is never right, even in the moments of the day when the time happens to be what it shows, because you don't gain any information about what time it is by looking at the clock.
This reasoning also applies to chatbots. If you can't tell whether what you have been given is useful information unless you alreay know the information, then you haven't been given useful information.
@riley
Yes, finally someone else gets it! -
@hypolite @riley
A computer is not a human, but tools can replace humans to do certain job if not better.
If you don't like dishwashers, laundry machines, sewing machines, tractors and diggers then by all means hire someone to do it, but most of us find it more effective to use machines instead
I would rather focus my time on building more complex things than waste it on doing less complex jobs that a machine (or AI) can easily do in less time@samir Nobody ever told me to treat my dishwasher as an employee, though, why do you feel compelled to do this with LLM-based AI systems?
And if the benefits of these systems were that clear and on par with previously established machines, we wouldn't have this kind of conversation. The problem still isn't that people are using them wrong.
