CIRCLE WITH A DOT

@Aedius @benjamineskola credit where it is due, I more or less stole this explanation from Dr. Emily Bender. Recommend checking out her podcast with Dr. Alex Hanna, where they roast Generative AI nonsense in the vein of Mystery Science Theater 3000. It is cathartic.

The Mystery AI Hype Theater 3000 Podcast

Our biweekly podcast deflates AI hype and draws attention to the real harms of the automation technologies we call "artificial intelligence".

DAIR (Distributed AI Research Institute) (dair-institute.org)

(I am not a linguist so please don't take my comments on language as authoritative.)

@benjamineskola @Beatpoet13 Yes, when humans see syntactically valid language, they assume there is a sentient intelligence on the other end. So when a machine generates this text, our brains go "the LLM is insightful."

The classic example is the ELIZA Effect, where people fell for this with a computer from 1966.

ELIZA effect - Wikipedia

(en.wikipedia.org)

"users [...] began to ascribe understanding and motivation to the program's output"

@benjamineskola @Beatpoet13 it can be helpful to replace an llm with "repeatedly pressing the suggested word on your phone keyboard."

If it spits out "I am a funny hamster", you wouldn't say it lied.

Humans are just not conditioned --- not wired, frankly --- to engage with machine generated, syntactically valid text. We suck at it. The ELIZA Effect wins every time.

@Aedius @benjamineskola language consists of two parts, the form and the sign

the form is the "tangible" part of the language, e.g. this text, or whatever soundwave physics bullshit is happening when we talk

the sign is the meaning, what you might visualize in your head when you read the word "cat"

LLMs only have access to form, so when the meaning of text is important (read: always), LLMs are not very useful

to this, promptfondlers always reply "but today is the worst it's ever gonna be"