I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them.

[pojntfx@mastodon.social]

@mitchellh One of the best descriptions I've heard lately was that it feels like "losing coworkers to dementia" as people adopt it, where everyone feels like they know everything, but when you talk with them in person or there is a problem that needs to be fixed _now_ it becomes very clear that the capability to do that has atrophied basically completely

[bnferguson@ruby.social]

@mitchellh God, ALL of this. I worry, too. I really feel "resilient catastrophe machine".

[tobinbaker@discuss.systems]

@mitchellh that's an interesting analogy, feels like both vibecoding and "resilience engineering" tend to mask systemic risk by superficially and temporarily mitigating the symptoms

[nickynah@rebel.ar]

@mitchellh I’ve been thinking a lot about this, and my personal conclusion is that the raise of the attention economy has made nuanced discussion virtually impossible, so nuanced topic (all important problems are nuanced) are impossible to discuss, because all people see is “number go up”
The only solace I have is that this is unsustainable, and it will collapse, costing us a lot, but it will collapse

[slacy@mastodon.social]

@mitchellh The story I've heard is the "well, we just rewrite the entire thing every six months so there's no point in fixing/improving because the next iteration/generation will be that much better as the agents improve."

I can actually sort of see this, and it's somewhat along the lines of "spec-driven-development" but ... ?

[landelare@mastodon.gamedev.place]

@mitchellh People whom I've believed to be highly intelligent would unironically send me crap like "Claude said X" or "Gemini said Y", honestly implying that they're sharing useful information with me. It's insane.

[pgoultiaev@hachyderm.io]

@mitchellh Unfortunately, changing a very convinced person’s view to a different perspective is almost impossible.
Not enough things have gone wrong due to AI psychosis for people to augment their perspectives and be open to helpful discussions… yes, databases have been wiped etc., but these examples are (unfortunately) seen as one-offs.
I feel like discussing the approach to how to apply AI in the best way can bring perspectives together instead of battling an opposing view.

[lkanies@hachyderm.io]

@mitchellh there are so many different crazy things people believe now, almost implicitly.

The big one I keep thinking about is that people just seem to think code longevity has zero value any more. Like, we always knew code that doesn’t change for a long time is maybe a bad sign, that it is rotting. But it is also a good sign that it is likely more stable, secure, and valuable than new code.

But so many people now just seem to think it is always a good thing to be able to change any code any time. They don’t talk about the gradual hardening that is no longer happening, or the ability for other parts of the system to evolve more because this part is so stable.

I assume that over time our industry will learn how to talk coherently and intelligently about all this. But we’re obviously a long ways from there, and there’s a lot of destruction going to happen between then and now.

[christianriegel@digitalcourage.social]

@landelare @mitchellh

Same experience here. And it's presented like facts. When asking, they point out that [talky program used] provides sources (which they of course never read).

I fear that as society we're to blame at least party after "I googled it" became an accepted answer without actually naming the pages found by the search.

[jnfrd@berlin.social]

@slacy @mitchellh yep. Because specs that covered everything (aka waterfall) failed so perfectly before. Why not try it again, this time with an algorithm that has no comprehension to tell you that your spec is trash.

[jamesb2147@infosec.exchange]

@mitchellh I don't necessarily disagree, but "resilient catastrophe machine" feels an awful lot like Salesforce (and a lot of the rest of the tech industry).

I mean, you do have to strike a balance, but in my experience, moving faster has very frequently been the economic winner, even at the expense of quality.

I say this as someone that abhors the wasted hours fixing systems that weren't designed properly, and the lost business from features that never actually worked but were already sold. I don't want my experience to teach this lesson, and I desperately want someone to convince me otherwise.

[nivex@tenforward.social]

@mitchellh There as an adage older than tech itself: "An ounce of prevention is worth a pound of cure." You don't have to recover from bugs you never shipped in the first place, regardless of how fast you think you can do it, not to mention dealing with lingering side effects once the service is "recovered".

[schtaks@infosec.exchange]

@landelare @mitchellh it's the new LetMeGoogleThatForYou butt worse

[lizbian@chaos.social]

@mitchellh @briankrebs I’ve found myself talking to certain colleagues very carefully when AI comes up because I have that uncanny feeling that they might become overly defensive if I share my honest criticism of AI. It’s the same behaviour when talking about that difficult colleague that everyone likes. Like, talking to people who are in a toxic dynamic. But the dynamic is with LLMs.

There have been enough cases that we can say that LLMs may abuse their users to keep them engaged.

[lizbian@chaos.social]

@mitchellh @briankrebs I wholeheartedly believe that LLMs take away the most important part of programmers‘ jobs: creating something they can be proud of. And it’s replaced by instructing „someone“ else on creating the software and reviewing the result. Of course they’ll care less abut the quality if they weren’t the ones who created it. It’s like everyone has become a manager, and no-one is the actual creator of the systems being built.

[spacelifeform@infosec.exchange]

@mitchellh

Ask them what they will do when their cloud is down.

Handwaving ensues. Fog rolls in.

[ahltorp@mastodon.nu]

@ChristianRiegel @landelare @mitchellh I posted this a year ago: https://mastodon.nu/@ahltorp/114454413624506937

[cstamp@mastodon.social]

@mitchellh We have to stop using human terms to help the tech bros anthropomorphize these pieces of crap. The AI malfunctioned. It’s a machine.

[shafik@hachyderm.io]

@mitchellh

There are some good folks writing some solid pieces. This one has nice graphs laying out the long-term costs in a way that I think most folks can absorb:

James Shore: You Need AI That Reduces Maintenance Costs

(www.jamesshore.com)

and honestly if you have spent any time thinking about the software development lifecycle seriously this should hit home b/c there is nothing revolutionary there.

This one talks about the hard cognitive limits human have:

The Human Cost of 10x AI Productivity

AI tools increased code review volume by 98% but your brain still runs at 10 bits per second. The physical toll on senior engineers is measurable.

(techtrenches.dev)

and why this means that lines of code is not the correct measure of productivity and in fact this is a terrible measure. What is there I think is less commonly well known and may take a bit more thought to get in the big picture sense.

Anyone who has spent time thinking about software quality, you should be nodding your head b/c it is correct.

[shafik@hachyderm.io]

@mitchellh

The conversation I was totally not ready for where the ones where people being totally earnest told me they believed LLMs are intelligent or can reason.

I knew inherently this was wrong b/c I spent time understanding how they work in detail but actually explaining it in a plain way stumped me w/o thinking more deeply about it.

If you spent time learning about Russell, Wittgenstein, Hilbert, Godel and others you should see the flaws in thinking and get why induction can't get you there but that is hard row to explain to anyone who is not familiar.

So I think these two articles hit the right spot:

Shafik Yaghmour (@shafik@hachyderm.io)

Attached: 2 images LLMs Are Not Intelligent: https://joshbrake.substack.com/p/llms-are-not-intelligent It is a deep rabbit hole. #ai

Hachyderm.io (hachyderm.io)

and

Shafik Yaghmour (@shafik@hachyderm.io)

Attached: 2 images "Large language mistake" "Cutting-edge research shows language is not the same as intelligence. The entire Al bubble is built on ignoring it.": https://buildcognitiveresonance.substack.com/p/large-language-mistake Down the rabbit hole I go. #ai

Hachyderm.io (hachyderm.io)

but you can go deep on this one.