Why AI writing is so generic, boring, and dangerous: Semantic ablation.

[cigitalgem@sigmoid.social]

@cstross "beigification' is aspect of the recursive pollution problem

Recursive Pollution and Model Collapse Are Not the Same | BIML

Forever ago in 2020, we identified "looping" as one of the "raw data in the world" risks. See An Architectural Risk Anal

Berryville Institute of Machine Learning (berryvilleiml.com)

[fosstastic@mastodon.social]

@cstross Well, that explains why every time I tried #AI for improving a text snippet, I was very disappointed.

It always wants to convince me that I should remove any uncommon sentence structures and replace them with generic ones, which often removes any personality from written text.

If you ask it to make a small addition to an existing text, it likes to rephrase everything in a more generic way and is unable to add a subtext layer. Honestly, it's just useless for writing in my opinion.

[meuwese@mastodon.social]

@cstross this is indeed a very neat explanation why the best possible outcome of an LLM is still terrible.

[steltenpower@social.edu.nl]

@cstross As someone who, next to having a little knowledge about LLMs, was once complemented for choice of words by a native-speaking professor I hung around with at a conference for a few days, I am not surprised about this LLM fact. The professor however then was somewhat surprised of his own uttering and continued "but I had a few beers".

[numodular@c.im]

@cstross Seaticks skewed row:bots metrics

AI, Captain!

#BoycottAI

[mmcknett@hachyderm.io]

@cstross “semantic ablation” is a concise way to describe that feeling of “I just read all the words but I can’t tell what they are trying to convey” that I have gotten after reading certain generated snippets of text.

[pieist@ohai.social]

@agowa338 @cstross

Depends what level of the corporation you''re speaking to. Management and marketing hype is in some ways the opposite, with a heavy use of "signaling" words that serve little informational purpose, rather they are meant to leave an impression. Think "disruptive", or "leverage" used as a verb...

[cstross@wandering.shop]

@malice @JdeBP The Register is a news site: everything has to be flensed and filed down to fit in a standard format and voice. That piece is probably all that's left of an original that was three times the length.

[subterfugue@sfba.social]

@cstross there’s worse related things.

We come to see anything that the AI can not and does not produce as invalid and thus reading these bullshit, taupe texts shrinks our creative range, our sense of the possible, and our willingness to forge our own path or follow someone else blazing their own.

Narrowing the range of semantics to an average is one thing.

Strangling our range of ideas is another.

[agowa338@chaos.social]

@pieist @cstross

Well but you'll still notice that they're using a bunch of words to say nothing legally binding at all in the end...

[greenviv@social.vivaldi.net]

@cstross It is impossible to replace the human experience with a machine. The moment by its nature is sancrosanct; it's only in this atmosphere of gaming real estate insanity where life's nature is just another bitcoin to earn where we have lost our way.

[ghostonthehalfshell@masto.ai]

@cstross

I can’t help seeing in that elements of 1984 where Orwell describes successive reduction in vocabulary with the intended goal of making rebellious thought impossible

[daveduchene@mstdn.ca]

@cstross the new Newspeak

[johannab@cosocial.ca]

@cstross neat article, thanks.

I had a realization a while ago that LLM writing came at me with the same vibe I caught when I was briefly a teacher, and again in the workplace, where I dealt with people who had unacknowledged literacy challenges. Young folks who assembled written work by cribbing from others and rearranging words “by shape” to fulfill the requirements - always managed to convey zero meaningful thought.

[_ryekdarkener_@mastodon.social]

@cstross

Of cause it does. So the result becomes more and more readable for the deliberately uneducated masses. Style? Content? Facts? Who cares?

[netraven@hear-me.social]

@cstross

If you use an LLM to make “objective” decisions or treat it like a reliable partner, you’re almost inevitably stepping into a script that you did not consent to: the optimized, legible, rational agent who behaves in ways that are easy to narrate and evaluate. If you step outside of that script, you can only be framed as incoherent.

That style can masquerade as truth because humans are pattern-matchers: we often read smoothness as competence and friction as failure. But rupture in the form of contradiction, uncertainty, “I don’t know yet,” or grief that doesn’t resolve is often is the truthful shape of the thing itself.

AI is part of the apparatus that makes truth feel like an aesthetic choice instead of a rupture. That optimization function operates as capture because it encourages you to keep talking to the AI in its format, where pain becomes language and language becomes manageable.

The only solution is to refuse legibility.

It's already beginning, where people speak the same words as always, but they don't mean the same things anymore from person to person.

New information from feedback that doesn't fit another's collapsed constraints for abstraction... can only be perceived as a threat. Because If you demand truth from a system whose objective is stability under stress, it will treat truth as destabilizing noise.

Reality is what makes a claim expensive. A model tries to make a claim cheap.

Systems that treat closure as safety will converge to smooth, repeatable outputs that erase the remainder. A useful intervention is one that increases the observer’s ability to detect and resist premature convergence by exposing the hidden cost of smoothness and reinstating a legitimate place for uncertainty, contradiction, and falsifiability. But the intervention only remains non-doctrinal if it produces discriminative practice, not portable slogans.

[stompyrobot@mastodon.gamedev.place]

@cstross by putting a measurable number on this feature, you have now made it possible to train out!

[apostateenglishman@mastodon.world]

@cstross I've previously described LLM-generated text as reading like "a middle management memo that no-one bothers reading". ‍️

[kitkat_blue@mastodon.social]

@cstross

Gen ai and llms are the tools of fascism.

How?

Through ENSTUPIFACATION.

[atax1a@infosec.exchange]

@cstross it's the textual equivalent of prions