It's clear that AI assisted coding is dividing developers (welcome to the culture wars!).
-
@matt @dalias You are absolutely right, but here's the thing: Code review also does not prevent subtle bugs from creeping into the code base when humans wrote the code. Review is just one of the tools that ensure software quality.
This is to say that code written by LLMs and humans suffer from similar issues, require similar care and review and can fail in similar ways. There is more LLM code, though, and there are new challenges because scaling with LLMs works differently than with humans.
@hanshuebner @dalias Isn't it obvious, though, that the risks are higher when you have an LLM generate code statistically from a natural-language prompt, as opposed to writing the code and paying attention to every detail yourself?
-
@hanshuebner @dalias Isn't it obvious, though, that the risks are higher when you have an LLM generate code statistically from a natural-language prompt, as opposed to writing the code and paying attention to every detail yourself?
@matt @dalias Statistically, you will have more bugs because you have more software. But also, you can easily create tests, refactor and make executable requirements.
Making good software with LLM support is hard work and takes time. If you look at the stuff that people make with three prompts and then post to LinkedIn, you know what I mean.
A good program requires attention to detail, no matter what the tool does for you.
-
@hanshuebner @flooper @plexus And if your view of the world begins and ends with making money, as I admit is capitalist dogma, fair enough.
But producing code with LLMs - or using them for anything which needs to be correct - is deception (whether you're deceiving yourself or others) on a massive scale, on a par with crypto, Ponzi schemes, climate denial, etc.
(1/2)
-
@matt @dalias Statistically, you will have more bugs because you have more software. But also, you can easily create tests, refactor and make executable requirements.
Making good software with LLM support is hard work and takes time. If you look at the stuff that people make with three prompts and then post to LinkedIn, you know what I mean.
A good program requires attention to detail, no matter what the tool does for you.
@hanshuebner @dalias So then why do it with an LLM as opposed to the hard work of writing the code directly? Is it just to appease capital's irrational demands?
-
@hanshuebner @dalias So then why do it with an LLM as opposed to the hard work of writing the code directly? Is it just to appease capital's irrational demands?
-
@hanshuebner @flooper @plexus And if your view of the world begins and ends with making money, as I admit is capitalist dogma, fair enough.
But producing code with LLMs - or using them for anything which needs to be correct - is deception (whether you're deceiving yourself or others) on a massive scale, on a par with crypto, Ponzi schemes, climate denial, etc.
(1/2)
-
@hanshuebner @flooper @plexus And if your view of the world begins and ends with making money, as I admit is capitalist dogma, fair enough.
But producing code with LLMs - or using them for anything which needs to be correct - is deception (whether you're deceiving yourself or others) on a massive scale, on a par with crypto, Ponzi schemes, climate denial, etc.
(1/2)
Anthropomorphizing them (as many do, but I don't think you are) is a flawed view, but does provide one useful insight.
If one treats an LLM as a person, then the fundamental issue is:
They are a bullshit artist with a huge library. They do not have competence at anything except bullshitting, at which they are superb.
I agree that it's amazing that we can build a mechanical bullshit generator that's good enough to bypass most people's defenses.
-
It's clear that AI assisted coding is dividing developers (welcome to the culture wars!). I've seen a few blog posts now that talk about how some people just "love the craft", "delight in making something just right, like knitting", etc, as opposed to people who just "want to make it work". As if that explains the divide.
How about this, some people resent the notion of being a babysitter to a stochastic token machine, hastening their own cognitive decline. Some people resent paying rent to a handful of US companies, all coming directly out of the TESCREAL human extinction cult, to be able to write software. Some people resent the "worse is better" steady decline of software quality over the past two decades, now supercharged. Some people resent that the hegemonic computing ecosystem is entirely shaped by the logic of venture capital. Some people hate that the digital commons is walled off and sold back to us. Oh and I guess some people also don't like the thought of making coding several orders of magnitude more energy intensive during a climate emergency.
But sure, no, it's really because we mourn the loss of our hobby.
@plexus Because AI did not create a programming language, because AI did not create a compiler, because AI did not create a linker, AI can not create software.
-
@hanshuebner @dalias But then you have to spend time putting guardrails in place (e.g. comprehensive tests) to make sure the LLM doesn't do something wrong; using an LLM is rolling the dice, after all. Now, if you believe that one should always put maximal guardrails in place anyway even for human-written code, then I suppose the faster code generation could still be a net gain. But I'm not sure there's one correct answer to how much one should invest in guardrails (tests, types, lints, etc.).
-
@hanshuebner @flooper @plexus I work for a living and try to avoid dishonesty while doing so.
Since I understand that LLMs are fundamentally and inherently dishonest, that doesn't leave much wiggle room for me.
-
It's clear that AI assisted coding is dividing developers (welcome to the culture wars!). I've seen a few blog posts now that talk about how some people just "love the craft", "delight in making something just right, like knitting", etc, as opposed to people who just "want to make it work". As if that explains the divide.
How about this, some people resent the notion of being a babysitter to a stochastic token machine, hastening their own cognitive decline. Some people resent paying rent to a handful of US companies, all coming directly out of the TESCREAL human extinction cult, to be able to write software. Some people resent the "worse is better" steady decline of software quality over the past two decades, now supercharged. Some people resent that the hegemonic computing ecosystem is entirely shaped by the logic of venture capital. Some people hate that the digital commons is walled off and sold back to us. Oh and I guess some people also don't like the thought of making coding several orders of magnitude more energy intensive during a climate emergency.
But sure, no, it's really because we mourn the loss of our hobby.
@plexus and i also feel we should be standing in solidarity with other affected professions to form a unified front against all generative "ai"
stand together with artists, writers, journalists, translators, etc etc against this morally corrupt technology -
Anthropomorphizing them (as many do, but I don't think you are) is a flawed view, but does provide one useful insight.
If one treats an LLM as a person, then the fundamental issue is:
They are a bullshit artist with a huge library. They do not have competence at anything except bullshitting, at which they are superb.
I agree that it's amazing that we can build a mechanical bullshit generator that's good enough to bypass most people's defenses.
@jmax @flooper I think I'm with you. The difficult part of LLMs for code generation for me is that the bullshit is executable. I can and do dismiss AI "prose", "art" and "music" easily because it is devoid of what makes me want to consume the thing in the first place. Code is primarirly consumed by machines, however, and its primary purpose is the functionality that it provides. That sets it apart from other slop.
-
@hanshuebner @flooper @plexus I work for a living and try to avoid dishonesty while doing so.
Since I understand that LLMs are fundamentally and inherently dishonest, that doesn't leave much wiggle room for me.
-
@hanshuebner @dalias But then you have to spend time putting guardrails in place (e.g. comprehensive tests) to make sure the LLM doesn't do something wrong; using an LLM is rolling the dice, after all. Now, if you believe that one should always put maximal guardrails in place anyway even for human-written code, then I suppose the faster code generation could still be a net gain. But I'm not sure there's one correct answer to how much one should invest in guardrails (tests, types, lints, etc.).
@hanshuebner For example, I write in Rust. I find that I never again want to do without the strong static typing, the controlled mutability, and the borrow checker. But @dalias writes excellent C code without these things. Would I trust an LLM to write C code like Rich does? Never. My point is that if the code is written by skilled humans, you don't necessarily need guardrails to the extent that you do for LLM-extruded code. So do LLMs really save time, *for high-quality code*? I'm skeptical.
-
@hanshuebner What does "software is better" even mean in this context?
I wonder if this entire "LLM generated code is good enough and it's creation is much more efficient" argument will stand the test of time when a lot of code is generated on the same product / project by many people. We do not know the answer to this yet.
@grishka@schaueho @hanshuebner @grishka
1/2
> "LLM generated code is good enough and it's creation is much more efficient" argument will stand the test of time when a lot of code is generated on the same product / project by many people. We do not know the answer to this yet.
I do suspect that some of the divide we see in the debate on Mastodon relates to the fact that some of the people arguing against it have not used LLMs to assist in writing Very Good Software At Scale using the methodologies available today.
I ship software to ~6 million monthly active users, with confidence, using Claude Code. I haven't written code by hand in ~10 months.
So, to the "We do not know the answer to this yet.", I think that we do.
We know that LLMs, used naively, make mistakes. And a craftsperson who knows the limitation of their tools (LLMs) can mitigate and verify in a number of ways.
Concession: LLMs were not ethically trained. Data centers are having awful impact on the energy grid + water use. I will never begrudge a person's choice to boycott.
Counterpoint: today, LLMs running on e.g. Apple Silicon approach the performance of the SOTA models. We're gonna see more of this, which will mitigate the individual's environmental impact as well as the need to pay forever-rent to big tech.
-
@hanshuebner @dalias But then you have to spend time putting guardrails in place (e.g. comprehensive tests) to make sure the LLM doesn't do something wrong; using an LLM is rolling the dice, after all. Now, if you believe that one should always put maximal guardrails in place anyway even for human-written code, then I suppose the faster code generation could still be a net gain. But I'm not sure there's one correct answer to how much one should invest in guardrails (tests, types, lints, etc.).
@matt @dalias In my own anecdotal and personal experience, LLM code gets enough things right to be competitive, but that experience is just a couple of months old. I can say, though, that the systems I created with LLM help very much fulfilled a real purpose, did not break randomly and are maintainable with LLM help.
-
@hanshuebner For example, I write in Rust. I find that I never again want to do without the strong static typing, the controlled mutability, and the borrow checker. But @dalias writes excellent C code without these things. Would I trust an LLM to write C code like Rich does? Never. My point is that if the code is written by skilled humans, you don't necessarily need guardrails to the extent that you do for LLM-extruded code. So do LLMs really save time, *for high-quality code*? I'm skeptical.
@matt @dalias One part of the conversation is of course the craftmanship - You write high-quality code as a matter of your ethos, and you employ the tools that you believe help you do that best. While other developers can be the judge of that, your users really cannot. To them, it is the external behavior of your code that matters.
Now, you can argue how user satisfaction is possible only with high-quality code, but that'd be mostly a theoretical discussion because most code in existence 1/
-
@schaueho @hanshuebner @grishka
1/2
> "LLM generated code is good enough and it's creation is much more efficient" argument will stand the test of time when a lot of code is generated on the same product / project by many people. We do not know the answer to this yet.
I do suspect that some of the divide we see in the debate on Mastodon relates to the fact that some of the people arguing against it have not used LLMs to assist in writing Very Good Software At Scale using the methodologies available today.
I ship software to ~6 million monthly active users, with confidence, using Claude Code. I haven't written code by hand in ~10 months.
So, to the "We do not know the answer to this yet.", I think that we do.
We know that LLMs, used naively, make mistakes. And a craftsperson who knows the limitation of their tools (LLMs) can mitigate and verify in a number of ways.
Concession: LLMs were not ethically trained. Data centers are having awful impact on the energy grid + water use. I will never begrudge a person's choice to boycott.
Counterpoint: today, LLMs running on e.g. Apple Silicon approach the performance of the SOTA models. We're gonna see more of this, which will mitigate the individual's environmental impact as well as the need to pay forever-rent to big tech.
@schaueho @hanshuebner @grishka
I will never begrudge a person's decision to boycott LLM usage.
But I do grow weary of folk on Mastodon earnestly insisting that "the flaws in LLMs will somehow all be laid bare, and handcrafted, artisanal code is somehow inherently superior"
Y'all cheering for John Henry without understanding that this is a job that's actually very well suited for a machine.
-
@matt @dalias In my own anecdotal and personal experience, LLM code gets enough things right to be competitive, but that experience is just a couple of months old. I can say, though, that the systems I created with LLM help very much fulfilled a real purpose, did not break randomly and are maintainable with LLM help.
@hanshuebner @matt Your anecdotes are contrary to empirical evidence and the mechanisms of how the thing works. I think at this point we can say you've bought into the parlor tricks and there's not much point in having this conversation.
-
@jmax @flooper I think I'm with you. The difficult part of LLMs for code generation for me is that the bullshit is executable. I can and do dismiss AI "prose", "art" and "music" easily because it is devoid of what makes me want to consume the thing in the first place. Code is primarirly consumed by machines, however, and its primary purpose is the functionality that it provides. That sets it apart from other slop.
@hanshuebner @flooper And the assumption that it's OK to build high rise apartments from paper mache, which is what I'm being asked to swallow, is not OK.
And the fact that we have a sophisticated machine for patching together buildings from recycled concrete slabs patched together with paper mache - carefully concealed where possible, or skillfully painted with stucco where necessary - makes it worse, not better.
Even if they do stand up for a little while before they collapse.
