I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them.
-
@mitchellh People whom I've believed to be highly intelligent would unironically send me crap like "Claude said X" or "Gemini said Y", honestly implying that they're sharing useful information with me. It's insane.
@landelare @mitchellh it's the new LetMeGoogleThatForYou butt worse
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh @briankrebs I’ve found myself talking to certain colleagues very carefully when AI comes up because I have that uncanny feeling that they might become overly defensive if I share my honest criticism of AI. It’s the same behaviour when talking about that difficult colleague that everyone likes. Like, talking to people who are in a toxic dynamic. But the dynamic is with LLMs.
There have been enough cases that we can say that LLMs may abuse their users to keep them engaged.
-
@mitchellh @briankrebs I’ve found myself talking to certain colleagues very carefully when AI comes up because I have that uncanny feeling that they might become overly defensive if I share my honest criticism of AI. It’s the same behaviour when talking about that difficult colleague that everyone likes. Like, talking to people who are in a toxic dynamic. But the dynamic is with LLMs.
There have been enough cases that we can say that LLMs may abuse their users to keep them engaged.
@mitchellh @briankrebs I wholeheartedly believe that LLMs take away the most important part of programmers‘ jobs: creating something they can be proud of. And it’s replaced by instructing „someone“ else on creating the software and reviewing the result. Of course they’ll care less abut the quality if they weren’t the ones who created it. It’s like everyone has become a manager, and no-one is the actual creator of the systems being built.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
-
Same experience here. And it's presented like facts. When asking, they point out that [talky program used] provides sources (which they of course never read).
I fear that as society we're to blame at least party after "I googled it" became an accepted answer without actually naming the pages found by the search.
@ChristianRiegel @landelare @mitchellh I posted this a year ago: https://mastodon.nu/@ahltorp/114454413624506937
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh We have to stop using human terms to help the tech bros anthropomorphize these pieces of crap. The AI malfunctioned. It’s a machine.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
There are some good folks writing some solid pieces. This one has nice graphs laying out the long-term costs in a way that I think most folks can absorb:
and honestly if you have spent any time thinking about the software development lifecycle seriously this should hit home b/c there is nothing revolutionary there.
This one talks about the hard cognitive limits human have:
The Human Cost of 10x AI Productivity
AI tools increased code review volume by 98% but your brain still runs at 10 bits per second. The physical toll on senior engineers is measurable.
(techtrenches.dev)
and why this means that lines of code is not the correct measure of productivity and in fact this is a terrible measure. What is there I think is less commonly well known and may take a bit more thought to get in the big picture sense.
Anyone who has spent time thinking about software quality, you should be nodding your head b/c it is correct.
-
There are some good folks writing some solid pieces. This one has nice graphs laying out the long-term costs in a way that I think most folks can absorb:
and honestly if you have spent any time thinking about the software development lifecycle seriously this should hit home b/c there is nothing revolutionary there.
This one talks about the hard cognitive limits human have:
The Human Cost of 10x AI Productivity
AI tools increased code review volume by 98% but your brain still runs at 10 bits per second. The physical toll on senior engineers is measurable.
(techtrenches.dev)
and why this means that lines of code is not the correct measure of productivity and in fact this is a terrible measure. What is there I think is less commonly well known and may take a bit more thought to get in the big picture sense.
Anyone who has spent time thinking about software quality, you should be nodding your head b/c it is correct.
The conversation I was totally not ready for where the ones where people being totally earnest told me they believed LLMs are intelligent or can reason.
I knew inherently this was wrong b/c I spent time understanding how they work in detail but actually explaining it in a plain way stumped me w/o thinking more deeply about it.
If you spent time learning about Russell, Wittgenstein, Hilbert, Godel and others you should see the flaws in thinking and get why induction can't get you there but that is hard row to explain to anyone who is not familiar.
So I think these two articles hit the right spot:
Shafik Yaghmour (@shafik@hachyderm.io)
Attached: 2 images LLMs Are Not Intelligent: https://joshbrake.substack.com/p/llms-are-not-intelligent It is a deep rabbit hole. #ai
Hachyderm.io (hachyderm.io)
and
Shafik Yaghmour (@shafik@hachyderm.io)
Attached: 2 images "Large language mistake" "Cutting-edge research shows language is not the same as intelligence. The entire Al bubble is built on ignoring it.": https://buildcognitiveresonance.substack.com/p/large-language-mistake Down the rabbit hole I go. #ai
Hachyderm.io (hachyderm.io)
but you can go deep on this one.
-
There are some good folks writing some solid pieces. This one has nice graphs laying out the long-term costs in a way that I think most folks can absorb:
and honestly if you have spent any time thinking about the software development lifecycle seriously this should hit home b/c there is nothing revolutionary there.
This one talks about the hard cognitive limits human have:
The Human Cost of 10x AI Productivity
AI tools increased code review volume by 98% but your brain still runs at 10 bits per second. The physical toll on senior engineers is measurable.
(techtrenches.dev)
and why this means that lines of code is not the correct measure of productivity and in fact this is a terrible measure. What is there I think is less commonly well known and may take a bit more thought to get in the big picture sense.
Anyone who has spent time thinking about software quality, you should be nodding your head b/c it is correct.
@shafik @mitchellh A cursory reading of Robert Pirsig's "Zen and the Art of Motorcycle Maintenance" betrays how quality has its own ineffable quality. KLOCs are not a useful metric often; McCabe's Cyclomatic Complexity is where I'd start. Dijkstra brought the wisdom yet as Adam Smith has been selectively quoted in econo-political argument, Dijsktra seems to be selectively quoted by members of the deep-learning community.
-
@mitchellh We have to stop using human terms to help the tech bros anthropomorphize these pieces of crap. The AI malfunctioned. It’s a machine.
@CStamp @mitchellh Kate Crawford warns about this ~7 pages into "Atlas of AI" and @HalvarFlake covered the topic well last July: https://addxorrol.blogspot.com/2025/07/a-non-anthropomorphized-view-of-llms.html
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh @DukeDuke hadn't occurred to me that the AI psychosis may be a factor driving the enshittification of tech, but that makes perfect sense. I swear, COVID and genAI are our civilization's answer to Romans' lead pipes updated for the 21st century
-
@shafik @mitchellh A cursory reading of Robert Pirsig's "Zen and the Art of Motorcycle Maintenance" betrays how quality has its own ineffable quality. KLOCs are not a useful metric often; McCabe's Cyclomatic Complexity is where I'd start. Dijkstra brought the wisdom yet as Adam Smith has been selectively quoted in econo-political argument, Dijsktra seems to be selectively quoted by members of the deep-learning community.
any metrics that solely look at the code will not tell you what you need to know.
The whole lifecycle has to be measured, including code review, bug reports, rates of errors over time etc
-
@mitchellh @briankrebs I wholeheartedly believe that LLMs take away the most important part of programmers‘ jobs: creating something they can be proud of. And it’s replaced by instructing „someone“ else on creating the software and reviewing the result. Of course they’ll care less abut the quality if they weren’t the ones who created it. It’s like everyone has become a manager, and no-one is the actual creator of the systems being built.
@lizbian @mitchellh @briankrebs Regarding the anthropomorphization argument: merely calling into question the nomenclature of "AI Alignment" is enough to trigger contempt/threat reactions in some "AI" proponents, betraying it has developed a pseudo-religious property. When people are unable to step to one side and consider the technology in its wider context, regardless of whether they were "one-shotting" code subsystems with e.g. GLM 5.1, demons form.
-
@mitchellh People whom I've believed to be highly intelligent would unironically send me crap like "Claude said X" or "Gemini said Y", honestly implying that they're sharing useful information with me. It's insane.
@landelare @mitchellh This is precisely why I prefix any and all LLM-originated outputs in my own working notes with "Parrot (Model name)" and believe me, more than once, I've caught e.g. Claude family models making wholly inappropriate Python API suggestions, betraying the model was mean-reverting to training set in its output token stream.
-
@mitchellh I’ve been thinking a lot about this, and my personal conclusion is that the raise of the attention economy has made nuanced discussion virtually impossible, so nuanced topic (all important problems are nuanced) are impossible to discuss, because all people see is “number go up”
The only solace I have is that this is unsustainable, and it will collapse, costing us a lot, but it will collapse@nickynah @briankrebs @mitchellh I have but dipped into Sokal and Bricmont's "Fashionable Nonsense: Postmodern Intellectuals' Abuse of Science" from 1999 on this point... the cult of market fundamentalism keeps betraying itself, and the "AI" pseudo-religion may yet cause capitalism to eat itself alive. This is where the existential risk lies. Not "AGI" or "ASI" fairy stories.
-
@mitchellh Unfortunately, changing a very convinced person’s view to a different perspective is almost impossible.
Not enough things have gone wrong due to AI psychosis for people to augment their perspectives and be open to helpful discussions… yes, databases have been wiped etc., but these examples are (unfortunately) seen as one-offs.
I feel like discussing the approach to how to apply AI in the best way can bring perspectives together instead of battling an opposing view.@pgoultiaev @mitchellh The updated EU Product Liability Directive 2025 explicitly assigns liability for induced, medically recognised, psychological damage, in the context of "AI" systems. There is a pseudo-religious sociological phenomenon occurring with regards to this technology. Neil Postman warned about this in "Technopoly". Ivan Illich proposed social strategies for actually dealing with and preventing it...
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh I have people I’ve been (or am) *very* close to I’ve had to say “we don’t talk about AI” with.
-
@lizbian @mitchellh @briankrebs Regarding the anthropomorphization argument: merely calling into question the nomenclature of "AI Alignment" is enough to trigger contempt/threat reactions in some "AI" proponents, betraying it has developed a pseudo-religious property. When people are unable to step to one side and consider the technology in its wider context, regardless of whether they were "one-shotting" code subsystems with e.g. GLM 5.1, demons form.
@lizbian @mitchellh @briankrebs Prof. Michael Wooldridge has expressed clearly, for wider dissemination on his Royal Society lecture from February, many of the problems recapitulated in this thread; the zingers are 30m in or so and he flies cover very carefully at the start. This is where I'd point civilians. But the odd Condescending Wonka meme may help. https://www.youtube.com/watch?v=CyyL0yDhr7I
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh Were you alive in the 1970's when GM's Van Nuys plant produced so many malfunctioning cars that they couldn't repair them fast enough to keep them from piling up on the lot there in Van Nuys? We'll fix them later, just get them off the line was the mantra. But they didn't. Fix them later. Sound familiar? Yes, but this time it will be different. Riiiiight.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh I used Claude today to verify a change I'd made to a piece of Python code. Instead of using a statically defined tuple of valid subclass names, it suggested using a pattern I found later on StackOverflow that it stated was "drift-proof" (which is what I wanted, to reduce toil in the code base.)
It offered me something that 1) failed right away due a missing import it didn't mention in its suggestion and 2) the code it suggested returned an empty tuple when I used the suggested pattern. I shrugged, and reverted to the manual method I had originally written.
Claude is arguably the most robust model for software engineering and I find it does dumb shit _all the fucking time_ ... and so many people just blithely assume it's right without checking.
