It's clear that AI assisted coding is dividing developers (welcome to the culture wars!).
-
@hanshuebner What does "software is better" even mean in this context?
I wonder if this entire "LLM generated code is good enough and it's creation is much more efficient" argument will stand the test of time when a lot of code is generated on the same product / project by many people. We do not know the answer to this yet.
@grishka@schaueho @hanshuebner @grishka
1/2
> "LLM generated code is good enough and it's creation is much more efficient" argument will stand the test of time when a lot of code is generated on the same product / project by many people. We do not know the answer to this yet.
I do suspect that some of the divide we see in the debate on Mastodon relates to the fact that some of the people arguing against it have not used LLMs to assist in writing Very Good Software At Scale using the methodologies available today.
I ship software to ~6 million monthly active users, with confidence, using Claude Code. I haven't written code by hand in ~10 months.
So, to the "We do not know the answer to this yet.", I think that we do.
We know that LLMs, used naively, make mistakes. And a craftsperson who knows the limitation of their tools (LLMs) can mitigate and verify in a number of ways.
Concession: LLMs were not ethically trained. Data centers are having awful impact on the energy grid + water use. I will never begrudge a person's choice to boycott.
Counterpoint: today, LLMs running on e.g. Apple Silicon approach the performance of the SOTA models. We're gonna see more of this, which will mitigate the individual's environmental impact as well as the need to pay forever-rent to big tech.
-
@hanshuebner @dalias But then you have to spend time putting guardrails in place (e.g. comprehensive tests) to make sure the LLM doesn't do something wrong; using an LLM is rolling the dice, after all. Now, if you believe that one should always put maximal guardrails in place anyway even for human-written code, then I suppose the faster code generation could still be a net gain. But I'm not sure there's one correct answer to how much one should invest in guardrails (tests, types, lints, etc.).
@matt @dalias In my own anecdotal and personal experience, LLM code gets enough things right to be competitive, but that experience is just a couple of months old. I can say, though, that the systems I created with LLM help very much fulfilled a real purpose, did not break randomly and are maintainable with LLM help.
-
@hanshuebner For example, I write in Rust. I find that I never again want to do without the strong static typing, the controlled mutability, and the borrow checker. But @dalias writes excellent C code without these things. Would I trust an LLM to write C code like Rich does? Never. My point is that if the code is written by skilled humans, you don't necessarily need guardrails to the extent that you do for LLM-extruded code. So do LLMs really save time, *for high-quality code*? I'm skeptical.
@matt @dalias One part of the conversation is of course the craftmanship - You write high-quality code as a matter of your ethos, and you employ the tools that you believe help you do that best. While other developers can be the judge of that, your users really cannot. To them, it is the external behavior of your code that matters.
Now, you can argue how user satisfaction is possible only with high-quality code, but that'd be mostly a theoretical discussion because most code in existence 1/
-
@schaueho @hanshuebner @grishka
1/2
> "LLM generated code is good enough and it's creation is much more efficient" argument will stand the test of time when a lot of code is generated on the same product / project by many people. We do not know the answer to this yet.
I do suspect that some of the divide we see in the debate on Mastodon relates to the fact that some of the people arguing against it have not used LLMs to assist in writing Very Good Software At Scale using the methodologies available today.
I ship software to ~6 million monthly active users, with confidence, using Claude Code. I haven't written code by hand in ~10 months.
So, to the "We do not know the answer to this yet.", I think that we do.
We know that LLMs, used naively, make mistakes. And a craftsperson who knows the limitation of their tools (LLMs) can mitigate and verify in a number of ways.
Concession: LLMs were not ethically trained. Data centers are having awful impact on the energy grid + water use. I will never begrudge a person's choice to boycott.
Counterpoint: today, LLMs running on e.g. Apple Silicon approach the performance of the SOTA models. We're gonna see more of this, which will mitigate the individual's environmental impact as well as the need to pay forever-rent to big tech.
@schaueho @hanshuebner @grishka
I will never begrudge a person's decision to boycott LLM usage.
But I do grow weary of folk on Mastodon earnestly insisting that "the flaws in LLMs will somehow all be laid bare, and handcrafted, artisanal code is somehow inherently superior"
Y'all cheering for John Henry without understanding that this is a job that's actually very well suited for a machine.
-
@matt @dalias In my own anecdotal and personal experience, LLM code gets enough things right to be competitive, but that experience is just a couple of months old. I can say, though, that the systems I created with LLM help very much fulfilled a real purpose, did not break randomly and are maintainable with LLM help.
@hanshuebner @matt Your anecdotes are contrary to empirical evidence and the mechanisms of how the thing works. I think at this point we can say you've bought into the parlor tricks and there's not much point in having this conversation.
-
@jmax @flooper I think I'm with you. The difficult part of LLMs for code generation for me is that the bullshit is executable. I can and do dismiss AI "prose", "art" and "music" easily because it is devoid of what makes me want to consume the thing in the first place. Code is primarirly consumed by machines, however, and its primary purpose is the functionality that it provides. That sets it apart from other slop.
@hanshuebner @flooper And the assumption that it's OK to build high rise apartments from paper mache, which is what I'm being asked to swallow, is not OK.
And the fact that we have a sophisticated machine for patching together buildings from recycled concrete slabs patched together with paper mache - carefully concealed where possible, or skillfully painted with stucco where necessary - makes it worse, not better.
Even if they do stand up for a little while before they collapse.
-
@hanshuebner @flooper Yes. But useful tools are those machines which do have honesty, in a mechanical sense.
-
@matt @dalias One part of the conversation is of course the craftmanship - You write high-quality code as a matter of your ethos, and you employ the tools that you believe help you do that best. While other developers can be the judge of that, your users really cannot. To them, it is the external behavior of your code that matters.
Now, you can argue how user satisfaction is possible only with high-quality code, but that'd be mostly a theoretical discussion because most code in existence 1/
@matt @dalias is not of high quality.
So we have an internal and an external view on quality that are not necessarily the same. At the same time, we have the external force for functionality, and I'd argue that to users, that force is more important than the internal quality of the code, which matters (only) to us.
The realization that with LLM help, people can create something that satisfies the desire of users in short amount of time will create more pull towards meeting those desires. 2/
-
@matt @dalias is not of high quality.
So we have an internal and an external view on quality that are not necessarily the same. At the same time, we have the external force for functionality, and I'd argue that to users, that force is more important than the internal quality of the code, which matters (only) to us.
The realization that with LLM help, people can create something that satisfies the desire of users in short amount of time will create more pull towards meeting those desires. 2/
@matt @dalias Saying that the desire can't be met because we can't create the software with the internal quality that we desire won't be successful in the long run.
Sure, some users will use bad software written with LLM help, blame it on LLMs and then ask for a handcrafted solution, if they can afford it. But that will be the exception, not the norm.
This is why I believe that as a software developer, I need to know how to work with LLMs rather than avoid them. YMMV. 3/3
-
@matt @dalias Saying that the desire can't be met because we can't create the software with the internal quality that we desire won't be successful in the long run.
Sure, some users will use bad software written with LLM help, blame it on LLMs and then ask for a handcrafted solution, if they can afford it. But that will be the exception, not the norm.
This is why I believe that as a software developer, I need to know how to work with LLMs rather than avoid them. YMMV. 3/3
@hanshuebner @dalias I'm sympathetic to the argument that creating more software and implementing more features faster isn't just about making money, but about solving problems, including problems that are causing suffering because a solution hasn't yet been implemented. I'm thinking in particular of the field that I work in, accessibility for disabled people. But programming via gambling, as one does when using an LLM, isn't the only way to address that urgency.
-
@hanshuebner @flooper And the assumption that it's OK to build high rise apartments from paper mache, which is what I'm being asked to swallow, is not OK.
And the fact that we have a sophisticated machine for patching together buildings from recycled concrete slabs patched together with paper mache - carefully concealed where possible, or skillfully painted with stucco where necessary - makes it worse, not better.
Even if they do stand up for a little while before they collapse.
-
@hanshuebner @dalias I'm sympathetic to the argument that creating more software and implementing more features faster isn't just about making money, but about solving problems, including problems that are causing suffering because a solution hasn't yet been implemented. I'm thinking in particular of the field that I work in, accessibility for disabled people. But programming via gambling, as one does when using an LLM, isn't the only way to address that urgency.
-
@hanshuebner @dalias I'm apprehensive about trying a full agent-based workflow because I'm afraid I'll be so dazzled by what it can do (unreliably) via brute force that I'll let my guard down in terms of evaluating it critically.
-
@hanshuebner @matt Yes it can hurt to try them. They are cognitohazards and are designed to make you think they're doing things they're not. This works on a lot of people, even people who think themselves very intelligent and thereby immune.
This is how we end up with folks praising them while putting out clearly worse writing, code, etc. that nobody wants.
-
It's clear that AI assisted coding is dividing developers (welcome to the culture wars!). I've seen a few blog posts now that talk about how some people just "love the craft", "delight in making something just right, like knitting", etc, as opposed to people who just "want to make it work". As if that explains the divide.
How about this, some people resent the notion of being a babysitter to a stochastic token machine, hastening their own cognitive decline. Some people resent paying rent to a handful of US companies, all coming directly out of the TESCREAL human extinction cult, to be able to write software. Some people resent the "worse is better" steady decline of software quality over the past two decades, now supercharged. Some people resent that the hegemonic computing ecosystem is entirely shaped by the logic of venture capital. Some people hate that the digital commons is walled off and sold back to us. Oh and I guess some people also don't like the thought of making coding several orders of magnitude more energy intensive during a climate emergency.
But sure, no, it's really because we mourn the loss of our hobby.
@plexus Let's not forget that code output by LLMs are probably not subject to copyright (and include training on copyrighted works), making them a legal landmine for any project or product that includes output from LLMs.
-
@hanshuebner @dalias I'm apprehensive about trying a full agent-based workflow because I'm afraid I'll be so dazzled by what it can do (unreliably) via brute force that I'll let my guard down in terms of evaluating it critically.
-
@hanshuebner @matt You realize you sound exactly like a drug dealer, right?
-
I have, and it failed to complete the task AND "lied" to me at the same time.
-
@schaueho @hanshuebner @grishka
I will never begrudge a person's decision to boycott LLM usage.
But I do grow weary of folk on Mastodon earnestly insisting that "the flaws in LLMs will somehow all be laid bare, and handcrafted, artisanal code is somehow inherently superior"
Y'all cheering for John Henry without understanding that this is a job that's actually very well suited for a machine.
@dusk
OT, but cool John Henry video: https://www.youtube.com/watch?v=kt9NSMZR0dM -
I have, and it failed to complete the task AND "lied" to me at the same time.
@BoydStephenSmithJr @grishka If you have the expectation that it should complete the task flawlessly and point out that it "lied", it seems that you have achieved your goal of showing that it did not work for you.
I've had many successes, and none of the things that I created magically collapsed or failed to work except under narrow circumstances. I had to spend time creating and improving them, but I would not have started them if I'd needed to write the code myself.
