I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh code rot
-
@mitchellh One of the best descriptions I've heard lately was that it feels like "losing coworkers to dementia" as people adopt it, where everyone feels like they know everything, but when you talk with them in person or there is a problem that needs to be fixed _now_ it becomes very clear that the capability to do that has atrophied basically completely
@pojntfx @mitchellh I have seen the same behavior even with capable engineers that in the past I knew as strong designers and code quality champions now reduced to faith followers with 2 or 3 agents working on different branches of the same project at the same time and really not understanding or owning anything and still feeling top performers.
I do agree it looks very much like a dementia -
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh I thought devs get chewed out for prod going down, and now management is normalizing prod going down? I feel like I first observed the change when Elon bought Twitter, and everyone was applauding his cost cutting and that Twitter only went down a couple times, when in the past every single downtime was significant.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh obviously written by a pissed sentient AI agent.....
-
I wonder how long it will take for the rest of the world to a) realize what's going on, b) arrive at the objectively valid conclusion and c) act on it and see it through.
Recently past experience (COVID-19) shows that being a tall order.
Anyway, I wonder when, if so, people who actively rejected contact with LLMs will be in extremely high demand, being knowledge bearers in teaching and technical lead positions.
@datenwolf @cczona @pojntfx The longer this goes on, the fewer of us will be left. And we're suffering in the meantime.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh MTBF vs MTTR is a symptom, not the disease.
The real question: when an AI-driven system causes harm, where does accountability live?
Failures get attributed to the AI, successes claimed by humans, liability dissolves into complexity.
We don't need humans to understand the system. We need governance architecture. Without that, MTTR is just failing faster and cleaner.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh I also constantly worry about „becoming one of them“ because I do think there are some cool use cases but they are not close to what some colleagues tell me it will be able to do (and I don’t see it happening) and it does not outweigh the negative impact in any moment
-
@mitchellh Unfortunately, changing a very convinced person’s view to a different perspective is almost impossible.
Not enough things have gone wrong due to AI psychosis for people to augment their perspectives and be open to helpful discussions… yes, databases have been wiped etc., but these examples are (unfortunately) seen as one-offs.
I feel like discussing the approach to how to apply AI in the best way can bring perspectives together instead of battling an opposing view.@pgoultiaev @mitchellh the issue with these „one-off“ failure cases is lack of empathy. They are seen as stupid mistakes by stupid people. I am smart, surely this won’t happen to me. So I don’t need to take any learnings from there, especially not questioning if this has any implications for the road I am travelling.
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
@mitchellh
Knowledge Management and Application Support guy here.What I see is a whole lot of faith-based planning that resembles the "innovation fairy" claptrap of years before. Companies are charging into the world while caving out experienced and trained people to save money in the belief, hope, prayer that the AI fairy will be there to catch them and build foundations under the causeway they are on.
There is no evidence or past performance to suggest the AI fairy will come through, and every indication that it won't
-
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
I agree with the broad brush strokes, but companies are not monolithic. For example, Management at my employer is completely AI-pilled, but my immediate team is much more sceptical about it, perhaps because we understand the technicalities better than Management does, and perhaps because it's hard to be enthusiastic about something that threatens wholesale job losses.
So there's tension within the organisation: pressure to adopt AI, good reasons not to, and a wide range of opinions represented.
-
@mitchellh I would love to see someone commission a study on this. It _feels_ like things are in general getting less reliable atm, esp. the stack I rely on for work (GitHub, Linear, Slack, Notion, VSCode, <insert-tui-tool-here>), but then I can't find any data on any of it.
Agreed. But I think it's worth distinguishing between two aspects of this decrease in reliability: one is the increasing use of #LLMs to write shoddy code that would previously have been written better by humans, but the other is the use of LLMs to find longstanding bugs in code written years ago by humans. The former is a real decrease in reliability; the other is illusory.
