Had a lot of fun with my stats students today.
-
FUN FACT: random ain't random. especially in computers.
if you ask for "random" output from a computer, there is no guarantee that what comes out isn't actually from the contents of RAM.
@burnitdown@beige.party You need to put some NDO into the ram. @futurebird@sauropods.win -
There is something very creepy about the way LLMs willy cheerfully give lists of "random" numbers. But they aren't random in frequency, and as my students pointed out "it's probably from some webpage about how to generate random numbers"
But even then, why is the frequency so unnaturally regular? Is that an artifact from mixing lists of real random numbers together?
@futurebird haven't tried it but maybe it's also all mixed up with non-random numbers in training content e.g. the next number after '20' is likely one of 0, 1 or 2, the start of a 21st century year so far. Or Benford's law https://en.wikipedia.org/wiki/Benford%27s_law
-
@burnitdown@beige.party You need to put some NDO into the ram. @futurebird@sauropods.win
@Life_is @futurebird that's still the contents of RAM, whatever an NDO is.
-
Had a lot of fun with my stats students today. I gave them two data sets. One from a random number generator, the other was one I made up that was not random, but designed to look random. They were able to figure out which one was fake.
Then we had ChatGPT make the same kind of data set (random numbers 1-6 set of 100) and it had the same problems as my fake set but in a different way.
We talked about the study about AI generated passwords.
@futurebird As you so often do, you sent me off on a tangent. My favorite PRNG is in Knuth, and it's called Algorithm A there. It is entirely additive, so very fast, and has a period of 2^54.
I spent *years* tryna to figure out why nobody ever used it or even mentioned it.
Finally discovered that it has another name, and that it is quite frequently used today.
I have, of course, completely forgotten its other name, which somebody here on fedi actually told me.
-
FUN FACT: random ain't random. especially in computers.
if you ask for "random" output from a computer, there is no guarantee that what comes out isn't actually from the contents of RAM.
@burnitdown @futurebird These days if you really want random numbers you can have them. Eg. RDRAND on Intel chips is seeded by analogue circuitry, not by some state updated in RAM. And even if you don't use RDRAND directly its output is still used as a source of entropy for other generators.
-
There is something very creepy about the way LLMs willy cheerfully give lists of "random" numbers. But they aren't random in frequency, and as my students pointed out "it's probably from some webpage about how to generate random numbers"
But even then, why is the frequency so unnaturally regular? Is that an artifact from mixing lists of real random numbers together?
@futurebird i mean the LLM itself is just a statistical distribution… the path through the distribution is i assume randomized, but the distribution itself is gonna be the same every time.
-
There is something very creepy about the way LLMs willy cheerfully give lists of "random" numbers. But they aren't random in frequency, and as my students pointed out "it's probably from some webpage about how to generate random numbers"
But even then, why is the frequency so unnaturally regular? Is that an artifact from mixing lists of real random numbers together?
The LLM is like a little box of computer horrors that we peer into from time to time.
I'm sorry but the whole interface is just so silly.
You ask for random numbers with sentences and it pretends to give them to you? What are we doooooing?
-
The LLM is like a little box of computer horrors that we peer into from time to time.
I'm sorry but the whole interface is just so silly.
You ask for random numbers with sentences and it pretends to give them to you? What are we doooooing?
@futurebird it really puts into perspective what my interaction with real people is like
-
The LLM is like a little box of computer horrors that we peer into from time to time.
I'm sorry but the whole interface is just so silly.
You ask for random numbers with sentences and it pretends to give them to you? What are we doooooing?
@futurebird It's very weird.
In principle, if you take an LLM, you should be able to get it to generate random numbers in a way that reflects the numbers that appear in the corpus it was trained on. If you have the raw model you can probably do that.
But if you ask ChatGPT (or at least if I do) it starts talking about how numbers taken from around us typically follow Benford's law so their first digits have a logarithmic distribution. When it then spits out some random numbers it's no longer sampling random numbers from the entire corpus but a sample that's probably heavily biased towards numbers that appear in articles about Benford's law. I.e. what people have previously said about these numbers, rather than the actual numbers.
-
The LLM is like a little box of computer horrors that we peer into from time to time.
I'm sorry but the whole interface is just so silly.
You ask for random numbers with sentences and it pretends to give them to you? What are we doooooing?
@futurebird as others here have said or implied, I think LLMs are trained not to be random. Like as a structural part of the statistical models they're based on, so the input corpus will inform the "random" output.
Speaking as a long time not mathematically rigorous enough amateur cryptographer, most humans don't understand (not talking about you or your students, to be clear) that actually random can contain sequences and patterns, or parts of them, so when an uninformed human evaluates "randomness", they don't recognize sequences with patterns even if those are accidental coincidences.
Related, there's also the old cryptography parable that if a low ranking person in the security organization uses random picking to draw random numbers for, for example, a one time pad, the results won't really be random if that volunteer looks into the hat or drum from which they pick because they will subconsciously bias toward patterns like letter and number frequency from their experience and expectations, which might help an attacker decrypt the pad. Maybe.
Since the LLM is supposed to emulate human output it makes sense it might mess with "randomness".
-
Which one is random?
(data sets are 100 numbers 1 to 6)listA=[2,3,5,1,2,2,4,2,4,5,2,3,3,4,5,6,4,2,6,2,2,1,3,4,5,5,6,3,3,6,1,4,2,1,4,5,2,2,3,3,3,5,6,3,2,4,5,5,1,1,1,6,1,4,3,5,5,3,1,1,1,6,1,4,6,6,3,6,6,2,4,4,4,5,1,5,6,2,6,1,1,2,4,2,2,3,4,4,5,6,1,3,3,3,5,4,6,5,1,6]
listB=[4,2,5,6,3,5,3,1,3,4,2,3,4,3,4,5,5,1,3,3,2,1,1,6,1,3,2,2,2,6,1,5,6,3,6,3,2,3,2,4,6,1,1,6,3,2,4,1,6,1,3,1,5,6,2,3,3,5,1,6,4,5,2,5,1,1,5,3,6,2,3,3,6,5,2,3,3,1,6,3,2,3,2,1,6,6,4,4,6,2,4,5,4,5,3,4,6,5,3,2]
-
Which one is random?
(data sets are 100 numbers 1 to 6)listA=[2,3,5,1,2,2,4,2,4,5,2,3,3,4,5,6,4,2,6,2,2,1,3,4,5,5,6,3,3,6,1,4,2,1,4,5,2,2,3,3,3,5,6,3,2,4,5,5,1,1,1,6,1,4,3,5,5,3,1,1,1,6,1,4,6,6,3,6,6,2,4,4,4,5,1,5,6,2,6,1,1,2,4,2,2,3,4,4,5,6,1,3,3,3,5,4,6,5,1,6]
listB=[4,2,5,6,3,5,3,1,3,4,2,3,4,3,4,5,5,1,3,3,2,1,1,6,1,3,2,2,2,6,1,5,6,3,6,3,2,3,2,4,6,1,1,6,3,2,4,1,6,1,3,1,5,6,2,3,3,5,1,6,4,5,2,5,1,1,5,3,6,2,3,3,6,5,2,3,3,1,6,3,2,3,2,1,6,6,4,4,6,2,4,5,4,5,3,4,6,5,3,2]
@futurebird @Bumblefish that question makes no sense
-
The LLM is like a little box of computer horrors that we peer into from time to time.
I'm sorry but the whole interface is just so silly.
You ask for random numbers with sentences and it pretends to give them to you? What are we doooooing?
@futurebird I am reminded of a Doctor Who episode, where they realize they are in a simulation because they are incapable of generating truly random numbers. One scene has a whole bunch of scientists sitting at a table and they all keep yelling the same number at the same time.
-
@burnitdown @futurebird These days if you really want random numbers you can have them. Eg. RDRAND on Intel chips is seeded by analogue circuitry, not by some state updated in RAM. And even if you don't use RDRAND directly its output is still used as a source of entropy for other generators.
@dpiponi@mathstodon.xyz @burnitdown@beige.party @futurebird@sauropods.win and cloudflare famously uses a camera pointed at a wall of lava lamps because the motion is random
-
Which one is random?
(data sets are 100 numbers 1 to 6)listA=[2,3,5,1,2,2,4,2,4,5,2,3,3,4,5,6,4,2,6,2,2,1,3,4,5,5,6,3,3,6,1,4,2,1,4,5,2,2,3,3,3,5,6,3,2,4,5,5,1,1,1,6,1,4,3,5,5,3,1,1,1,6,1,4,6,6,3,6,6,2,4,4,4,5,1,5,6,2,6,1,1,2,4,2,2,3,4,4,5,6,1,3,3,3,5,4,6,5,1,6]
listB=[4,2,5,6,3,5,3,1,3,4,2,3,4,3,4,5,5,1,3,3,2,1,1,6,1,3,2,2,2,6,1,5,6,3,6,3,2,3,2,4,6,1,1,6,3,2,4,1,6,1,3,1,5,6,2,3,3,5,1,6,4,5,2,5,1,1,5,3,6,2,3,3,6,5,2,3,3,1,6,3,2,3,2,1,6,6,4,4,6,2,4,5,4,5,3,4,6,5,3,2]
@futurebird @Bumblefish There's literally no way to say whether a list of numbers is random or not (1, 1, 1, 1, etc can plausibly be a random sequence for all we know), though you can establish likelihoods by looking at the distribution.
-
Which one is random?
(data sets are 100 numbers 1 to 6)listA=[2,3,5,1,2,2,4,2,4,5,2,3,3,4,5,6,4,2,6,2,2,1,3,4,5,5,6,3,3,6,1,4,2,1,4,5,2,2,3,3,3,5,6,3,2,4,5,5,1,1,1,6,1,4,3,5,5,3,1,1,1,6,1,4,6,6,3,6,6,2,4,4,4,5,1,5,6,2,6,1,1,2,4,2,2,3,4,4,5,6,1,3,3,3,5,4,6,5,1,6]
listB=[4,2,5,6,3,5,3,1,3,4,2,3,4,3,4,5,5,1,3,3,2,1,1,6,1,3,2,2,2,6,1,5,6,3,6,3,2,3,2,4,6,1,1,6,3,2,4,1,6,1,3,1,5,6,2,3,3,5,1,6,4,5,2,5,1,1,5,3,6,2,3,3,6,5,2,3,3,1,6,3,2,3,2,1,6,6,4,4,6,2,4,5,4,5,3,4,6,5,3,2]
@futurebird @Bumblefish The only way you could determine that something’s not random is if a pattern emerges in the data set. Even still, statistically, it is probable for a CSPRNG with good entropy to produce a random data set that looks like it’s not random—unlikely, but probable.
-
@futurebird It's very weird.
In principle, if you take an LLM, you should be able to get it to generate random numbers in a way that reflects the numbers that appear in the corpus it was trained on. If you have the raw model you can probably do that.
But if you ask ChatGPT (or at least if I do) it starts talking about how numbers taken from around us typically follow Benford's law so their first digits have a logarithmic distribution. When it then spits out some random numbers it's no longer sampling random numbers from the entire corpus but a sample that's probably heavily biased towards numbers that appear in articles about Benford's law. I.e. what people have previously said about these numbers, rather than the actual numbers.
@dpiponi Even with a raw model, I don't see how you would sample from the distribution of numbers in the corpus. Perhaps provide no context and sample one or more tokens (using an independent pseudo-random number generator) from the distribution, and if the returned token parses as a number, return it to the user, otherwise try again. Providing any context/prompt would bias what is returned. This seems too contrived/circular.
@futurebird -
@futurebird @Bumblefish There's literally no way to say whether a list of numbers is random or not (1, 1, 1, 1, etc can plausibly be a random sequence for all we know), though you can establish likelihoods by looking at the distribution.
You *can* make an argument for one of these lists being random like a dice roll and the other being much less likely to be generated in that way.
-
@futurebird @Bumblefish The only way you could determine that something’s not random is if a pattern emerges in the data set. Even still, statistically, it is probable for a CSPRNG with good entropy to produce a random data set that looks like it’s not random—unlikely, but probable.
Only one of these lists could *plausibly* be from rolling dice.
-
@futurebird @Bumblefish The only way you could determine that something’s not random is if a pattern emerges in the data set. Even still, statistically, it is probable for a CSPRNG with good entropy to produce a random data set that looks like it’s not random—unlikely, but probable.
@futurebird @Bumblefish I have a UUID-generating library that, under certain conditions, could generate the same identical UUIDs because the CSPRNG it used ended up reusing the same entropy seed, unless the server was restarted. That was a *fun* bug to investigate and fix.
