An insightful thread by @glyph

[n_dimension@infosec.exchange]

An insightful thread by @glyph

THE GRAND MASTODON THEATRE
presents
AN EVENING OF EPISTEMIC DISRUPTIONS

~ Starring Mr. GLYPH in reply to Mr. DOCTOROW ~

ACT I — THE RADIUM PAINT RIPOSTE
"Normal technology" ≠ "safe technology." We are in the pre-germ-theory age of AI: combating miasma with bloodletting whilst installing the miasma-generator in every home.

ACT II — THE BASILISK'S GAZE
Cornell study: biased AI autocomplete shifted people's beliefs — EVEN THOSE WHO REJECTED THE SUGGESTIONS. Warnings did nothing. If you can see it, the basilisk has already won.

ACT III — YES THEY BLOODY WELL ARE
"They're not vibe-coding mission-critical AWS modules." Amazon's AI tool deleted an entire AWS environment (13hr outage). 6hr retail site collapse in March. Senior sign-off now required. They ARE.

ACT IV — THE SYMBOLOID SLOT MACHINE
LLMs are epistemic disruptors by design. The chat UI is a slot machine: you remember jackpots, forget losses. Dismiss ALL subjective experience of LLM use. Fly by instruments.

ACT V — THE FUTZING FRACTION
FF = (W+I+C+E)/(P×H). Checking costs are always higher than you think. Success rate always lower. Breaks even at ~5 prompts per task under ABSURDLY optimistic assumptions. Nobody measures this.

GRAND FINALE — THE ASK
"This might be toxic brain poison. Show me data it's safe?" No study proves it IS. Several suggest it might be. Nothing in the way of a successful safety test. He'd love to be wrong. Still waiting.
@ EXITS ARE CLEARLY MARKED @
(though whether any of us can truly exit remains open)

Glyph (@glyph@mastodon.social)

I wish I could recommend this piece more, because it makes a bunch of great points, but the "normal technology" case feels misleading to me. It's not _wrong_, exactly, but radium paint was also a "normal technology" according to this rubric, and I still very much don't want to get any on me and especially not in my mouth

Mastodon (mastodon.social)