This has been my experience when using Claude Code too.
-
@daisy thanks for the mention! and yeah this is also really bad, and I am in the process of writing about it right now. to summarize briefly: you don’t just have to scrutinize every line once (you can’t do that, due to vigilance decrement), you also have to maintain the expertise forever (lots more lines of code = lots more maintenance burden)
@glyph Yeah. I have a gut feeling that we as an industry are barreling towards a maintenance crisis that we are going to have to reckon with, on account of LLMs’ tendencies towards unnecessary complexity and huge volumes of code. Like, there are a number of different possible futures where LLMs cannot realistically be used to maintain the code that they wrote, and if any of those futures come to pass, unfucking the mess is going to be an enormous undertaking.
-
@glyph Yeah. I have a gut feeling that we as an industry are barreling towards a maintenance crisis that we are going to have to reckon with, on account of LLMs’ tendencies towards unnecessary complexity and huge volumes of code. Like, there are a number of different possible futures where LLMs cannot realistically be used to maintain the code that they wrote, and if any of those futures come to pass, unfucking the mess is going to be an enormous undertaking.
-
@glyph Yeah. I have a gut feeling that we as an industry are barreling towards a maintenance crisis that we are going to have to reckon with, on account of LLMs’ tendencies towards unnecessary complexity and huge volumes of code. Like, there are a number of different possible futures where LLMs cannot realistically be used to maintain the code that they wrote, and if any of those futures come to pass, unfucking the mess is going to be an enormous undertaking.
-
RE: https://eigenmagic.net/@abstractcode/116525091915148748
This has been my experience when using Claude Code too. I feel like the time that it takes to rework its output matches or exceeds the time it would have taken to just write the code. And the issues I have to fix vary from code that will be difficult to maintain, to code that only works for a very narrow ideal case, to code that just plain will not do what it is supposed to do.
And that has really changed my perception of people who claim that it’s a huge productivity booster for them, to be honest. Like I don’t know if they’re using the tool heaps differently and getting better results, or they’re doing the same rework as I’m doing and just not perceiving it as a slowdown, or if they have different and much more permissive quality standards to mine.
@daisy @abstractcode
This still applies as well. In the end people just concede to some 80% solution, giving up fighting the prompts.
-
RE: https://eigenmagic.net/@abstractcode/116525091915148748
This has been my experience when using Claude Code too. I feel like the time that it takes to rework its output matches or exceeds the time it would have taken to just write the code. And the issues I have to fix vary from code that will be difficult to maintain, to code that only works for a very narrow ideal case, to code that just plain will not do what it is supposed to do.
And that has really changed my perception of people who claim that it’s a huge productivity booster for them, to be honest. Like I don’t know if they’re using the tool heaps differently and getting better results, or they’re doing the same rework as I’m doing and just not perceiving it as a slowdown, or if they have different and much more permissive quality standards to mine.
@daisy Couldn't they also be vastly slower programmers?
-
@glyph Yeah. I have a gut feeling that we as an industry are barreling towards a maintenance crisis that we are going to have to reckon with, on account of LLMs’ tendencies towards unnecessary complexity and huge volumes of code. Like, there are a number of different possible futures where LLMs cannot realistically be used to maintain the code that they wrote, and if any of those futures come to pass, unfucking the mess is going to be an enormous undertaking.
-
@kevingranade @daisy I think you’re just misreading this, nowhere did daisy suggest that the LLMs will be involved in the repair effort
-
@dalias @daisy we all need to be talking more about the social function LLMs are serving, too. enterprise code is not acausally or transcendentally like this. people want to do a good job. unrealistic enterprise expectations create working environments that facilitate repetitive code that is hard to understand. what LLMs are doing even beyond the structure of their output (which IS bad, don’t get me wrong) is encouraging the development of even *more* unrealistic expectations from management
-
@dalias @daisy we all need to be talking more about the social function LLMs are serving, too. enterprise code is not acausally or transcendentally like this. people want to do a good job. unrealistic enterprise expectations create working environments that facilitate repetitive code that is hard to understand. what LLMs are doing even beyond the structure of their output (which IS bad, don’t get me wrong) is encouraging the development of even *more* unrealistic expectations from management
@dalias @daisy the bot *looks like* it has done the job, so leadership decides the job is done. time for more jobs! now everybody has nine jobs, at least one of which is filling out TPS reports explaining why they shouldn’t be fired because they have embraced the AI that’s making them so much more productive. (this is, ironically, the only job that the AI is legitimately good at doing, because it is adversarial bullshit communication, the one thing LLMs excel at)
-
RE: https://eigenmagic.net/@abstractcode/116525091915148748
This has been my experience when using Claude Code too. I feel like the time that it takes to rework its output matches or exceeds the time it would have taken to just write the code. And the issues I have to fix vary from code that will be difficult to maintain, to code that only works for a very narrow ideal case, to code that just plain will not do what it is supposed to do.
And that has really changed my perception of people who claim that it’s a huge productivity booster for them, to be honest. Like I don’t know if they’re using the tool heaps differently and getting better results, or they’re doing the same rework as I’m doing and just not perceiving it as a slowdown, or if they have different and much more permissive quality standards to mine.
@daisy I am no expert at all but one of the first things I could base my arguments on when llms became the new cease was this study, apparently finding experienced devs feel they are 24% more productive when using llms while they are actually about 20% slower
-
> Like I don’t know if they’re using the tool heaps differently and getting better results, or they’re doing the same rework as I’m doing and just not perceiving it as a slowdown, or if they have different and much more permissive quality standards to mine.
I suspect it's a mixture of things, but IME the people who I work with directly that all see it as a force multiplier are way more permissive of quality (and arguably wouldn't know what quality even looks like anyway), and are also bad at tracking how long things take.
One guy I know spent 8 hours prompting and reprompting claude to craft a plan, then had it implement the plan. The plan was garbage and the implementation was worse than the plan. I could have cranked out what he had Claude do in less time. The code review was also a slog, took me longer to review the code than it took him to prompt claude (it was like 8k LOC), and there were like 200 comments on the PR. He spoke highly of this experience and described claude as a force multiplier afterwards. He's delusional.
it seems just about anything can be classified as "faster" if you simply choose not to count time spent on parts of the problem as "working on it"
someone recently "improved" the time to complete a task from ~2hr to ~4days but so long as the first 3.99 days can somehow be justified as "not counting", they can call it faster
i think it's half and half: there is pressure to deny reality, there is also the perennial inability of enterprise programmers to accurately estimate any task, past or future: i think some folks genuinely aren't perceiving the time spent
-
it seems just about anything can be classified as "faster" if you simply choose not to count time spent on parts of the problem as "working on it"
someone recently "improved" the time to complete a task from ~2hr to ~4days but so long as the first 3.99 days can somehow be justified as "not counting", they can call it faster
i think it's half and half: there is pressure to deny reality, there is also the perennial inability of enterprise programmers to accurately estimate any task, past or future: i think some folks genuinely aren't perceiving the time spent
@erisceleste @daisy Yes, as far as I can tell the guy I was dealing with was only measuring the time it took to write the code, and not all of the time around it. He was bragging about he only wrote "about 5 lines of code", and he was absolutely not counting my code review time, which was certainly more time than he worked on the whole PR.
-
R relay@relay.infosec.exchange shared this topic
