Had a lot of fun with my stats students today.

[meuwese@mastodon.social]

@ai6yr @ohmu @futurebird wait so... is that the ultimate question? "What number will an LLM always include when generating random numbers?"

[life_is@no-pony.farm]

@burnitdown@beige.party @futurebird@sauropods.win raNDOm. A play on words.

[okohll@hachyderm.io]

@cstross @futurebird God does play dice, but there’s a big lead weight in one side

[thisalex@hachyderm.io]

@futurebird
> what are we doing?

I think that the best description is, that we take part in a play. LLM makes its best effort to write how this dialogue could continue to look plausible for the reader. Choose your own adventure.

[mildouze@mamot.fr]

@futurebird @Bumblefish
B
(Random answer)

[lamecarlate@pouet.it]

@futurebird @Bumblefish I'm no stats student, so maybe I haven't the bases (for lack of a better term, English is not my main language), but I think listA is the random one. The fact that in the listB there is nearly no triplets seems too good to be true.

[ingalovinde@embracing.space]

@AbyssalRook @futurebird I see two mistakes in your reasoning.
One is technical: events "numbers with position N, N+1 and N+2 are the same" for different values of N are _not_ independent of each other. (For example, if we know that this statement is true for N=10, then there likelihood of it being true for N=11 is 1/6, not 1/36.)
Another symbolizes a deeper problem with a lot of modern research that relies heavily on p-values: consider how many statements of this kind, containing the same amount of information, could you make? Unless you commit to a specific statement beforehand, before seeing the data: "this statement would only be true in 8% of cases for truly random data" does not really mean anything if it's just one out of 20 equally "interesting" statements one could make about the data (e.g. "how many triplets of incrementing numbers (modulo six) are there", "how many decrementing triplets are there", etc), each only 8% likely. Because of course it is expected that for most random sequences, a few of these individually not very likely statements will be true.

[futurebird@sauropods.win]

@lamecarlate @Bumblefish

I've got some bad news. I've posted the solution with a CW on the original thread.

[futurebird@sauropods.win]

@IngaLovinde @AbyssalRook

It's been really helpful for me to see how many people focused on the order of the numbers in the list, which I didn't think very important since the list is so short that that type of analysis might not be that useful.

I used the random list to scramble the fake numbers twice. I should have scrambled them more.

[abyssalrook@mstdn.social]

@IngaLovinde I'm not following the first problem in the logic. The situation you're describing might be important if we're looking at more and more instances of it happening, but looking at it happening at least once (~94%) doesn't change at all, and it happening ONLY once might jiggle the ~8% estimate I had, but not significantly move it.

[flockofcats@famichiki.jp]

@Bumblefish @futurebird
That was an interesting thread. Our brains are wired to think certain things are “random” when they’re not, so when people try to create something that looks random, they often avoid repeated numbers, even though there’d be repeats, if truly random, with some expected frequency. Also, odd numbers are often overrepresented cuz they feel more random, e.g., 5973 vs 6084. This “ looks random, but isn’t” often comes up when people fabricate scientific data

[abyssalrook@mstdn.social]

@IngaLovinde As for the latter, that is entirely true from a research perspective, but I picked the 3-of-a-kind pattern because I assumed the non-random list was entirely human constructed, and that particular pattern is one that sticks out to us the most. Someone making a list by hand is more likely to see "6-6-6" as less random than "6-1-2" or "3-4-5".

I did not clock 'Which is random?' as one being a dice roll and the other being a shuffled deck of prescribed cards.

[fsologureng@chilemasto.casa]

@futurebird listA has the subsequence 1,1,1,6,1,4 repeated twice at very short distance between them, which is, while plausible, extremely improbable. That's the way I found it's crafted.

[demfighter@mas.to]

@futurebird In essence, an LLM is nothing more than a glorified and dumbed down search engine.

Instead of producing a set of hyperlinks like a normal search engine would, the algorithm takes excerpts from the sources with the highest "relevance" value. The output is formatted to look like pseudo-speech for no apparent reason.

The end result is never better than the traditional search results, which may or may not be useful. The only thing the LLMs are good at is wasting electricity.

[ingalovinde@embracing.space]

@AbyssalRook okay let's calculate it:
Let a_n be the probability that the sequence of length n does not contain triplets of identical numbers, and does not end with two same numbers; b_n, the same, but ends with two same numbers.
Then a_1 = 1, a_2 = 5/6, b_2 = 1/6; a_(n+1) = a_n * 5/6 + b_n * 5/6; b_(n+1) = a_n * 1/6.
Or, expanding b_n, we get a_(n+2) = a_(n+1) * 5/6 + a_n * 5/36.
Plugging these numbers into Wolfram alpha (`LinearRecurrence[{5/6, 5/36}, {1, 5/6}, 100]`), we obtain a_100 ~= 0.0762866, a_99 ~= 0.0781878, and therefore the probability that the sequence of 100 random numbers does not contain triplets of the same number is a_100 + a_99/6 ~= 0.0893 = 8.93%.

By contrast, the probability that out of 98 random (and independent) triplets none will consist of three same numbers is (35/36)^98 ~= 6.32%.

That's a pretty large difference, and not just a jiggle.

(I understand that this is not the number you were looking at, but it's the easiest way to illustrate that there is a significant difference between answering questions about triplets of repeating number among 98 independent random triplets and among 98 sub-triplets of the sequence with 100 independent random numbers.)

[ai6yr@m.ai6yr.org]

@meuwese @ohmu @futurebird Apparently humans have willed that into existence, yes. LOL. (err... Douglas Adams, precisely)

[ldpm@wandering.shop]

@futurebird @ramsey @Bumblefish this is not remotely my area of expertise but I am interested in the answer. My guess would be that the list that looks more evenly distributed is the fake one, and therefore List A is the "actually random" one because it has more seemingly outlying subsets, like a whole bunch of 1s in rapid succession.

There are tons of ways to unevenly distribute but relatively few ways to evenly distribute, so the one that seems less even is more likely to be true

[ldpm@wandering.shop]

@futurebird @ramsey @Bumblefish also I suspect maybe a Monty Hall kind of thing where you generated a bunch of random lists, and then selected the one that looked least random to you to trick your students.

I'd love to know what the actual answer is and what you were hoping to teach your students!

[raffzahn@mastodon.bayern]

@futurebird

"What are we doooooing?"

Well, we've taken the sound algorithm of a brabbling baby, supercharged by a huge library of words annotated by possibility of sequence and now management is jumping around like parents bragging what a genius their 11 month old is. All because WE try to find meaning in the perceived word sequence.

Same management that brags about 1400% lower prices :))

[raffzahn@mastodon.bayern]

@dpiponi @futurebird

Which in turn is what LLM do. They give an averaged output, not a reasoned.

In addition the inherent laws of measurement and control define that any reached output will never met the intended. Thus LLM output will never increase knowledge, but migrate toward zero.