Free software people: A major goal of free software is for individuals to be able to cause software to behave in the way they want it toLLMs: (enable that)Free software people: Oh no not like that
-
@promovicz I think a set of instructions to a machine should not be copyrightable and the rest flows from there.
@mjg59 Since you don't want to talk about the human/social side, you can only miss it.
-
@troed @chris_evelyn @mjg59 last time I checked, Mistral models were merely open weight, with no training dataset available nor training pipeline released as FOSS. Has that changed?
-
@mjg59 but you are paying the owner of the machine a recurring rent, aren't you? does this not bother you? what this machine does for you will never be yours, you will pay them again and again. you do not own the tools of your trade anymore. If the rent seeking owner denies you access or you can not afford it anymore this is all gone.
@mjg59 before all this computer stuff I learned a manual craft, I still own the skills, I can use them when I need them, no one has to be paid. I can gift these skills to people, that need but can not afford them otherwise. Is this not, what all is about?
-
Free software people: A major goal of free software is for individuals to be able to cause software to behave in the way they want it to
LLMs: (enable that)
Free software people: Oh no not like that@mjg59 heh, poked the bear with this one.
unfortunately most computer users are also not set up in a way where a subtle piece of malware injected into a python script or something could ruin their day.
tao uses them for generating proofs, which seems to work, and I have had some use to similar ends. they're quite bad if you have any delay in your feedback loop though
-
@david_chisnall @mjg59 @ignaloidas I have encountered plenty of people and books that were wrong, so I still have to engage my brain and double check, though.
@mnl @david_chisnall @mjg59 @ignaloidas
Yes, but that is also not the argument.
If you read a book to extract information, you already have a mental model of the failure modes (or can build one, like students do)
- Is the author known to be proficient in the space
- Is the publisher reputable
- Is the book 'new'Depending on the answers to those questions, you either take the content as absolutely correct, likely correct, plausible, or problematic. You can know those things before
1/2
-
@mjg59 but you are paying the owner of the machine a recurring rent, aren't you? does this not bother you? what this machine does for you will never be yours, you will pay them again and again. you do not own the tools of your trade anymore. If the rent seeking owner denies you access or you can not afford it anymore this is all gone.
@Nfoonf @mjg59 one, you're clearly not a carpenter if you think they don't pay rent, maintenance costs, overhead, professional licensing, etc. two, you don't understand how the industry works. production is a really big deal, even if you're a craftsman.
craftmanship has always been for the well-to-do. most people get buy with what they can get cheaply, which means sending everything with uncalibrated nailguns, close-enough tolerances and twisted studs here.
-
@Nfoonf @mjg59 one, you're clearly not a carpenter if you think they don't pay rent, maintenance costs, overhead, professional licensing, etc. two, you don't understand how the industry works. production is a really big deal, even if you're a craftsman.
craftmanship has always been for the well-to-do. most people get buy with what they can get cheaply, which means sending everything with uncalibrated nailguns, close-enough tolerances and twisted studs here.
-
@mnl @david_chisnall @mjg59 @ignaloidas
Yes, but that is also not the argument.
If you read a book to extract information, you already have a mental model of the failure modes (or can build one, like students do)
- Is the author known to be proficient in the space
- Is the publisher reputable
- Is the book 'new'Depending on the answers to those questions, you either take the content as absolutely correct, likely correct, plausible, or problematic. You can know those things before
1/2
@mnl @david_chisnall @mjg59 @ignaloidas
even reading the first page.
Generally, this assessment of the overall book extends to each page, even if it contains pages with errors.
For llms, there is a probability that each query is resulting in garbage. In the book-analogy, it is as if each page is written by a different author, some experts, some crooks
Except no page is attributed, and guessing who wrote what page is up to the reader.
There is no model to be build around that fail-mode
2/2 -
I’ve heard this argument before and I disagree with it. My goal for Free Software is to enable users, but that requires users have agency. Users being able to modify code to do what they want? Great! Users being given a black box that will modify their code in a way that might do what they want but will fail in unpredictable ways, without giving them any mechanism to build a mental model of those failure modes? Terrible!
I am not a carpenter but I have an electric screwdriver. It’s great. It lets me turn screws with much less effort than a manual one. There are a bunch of places where it doesn’t work, but that’s fine, I can understand those and use the harder-to-use tool in places where it won’t work. I can build a mental model of when not to use it and why it doesn’t work and how it will fail. I love building the software equivalent of this, things that let end users change code in ways I didn’t anticipate.
But LLM coding is not like this. It’s like a nail gun that has a 1% chance of firing backwards. 99% of the time, it’s much easier than using a hammer. 1% of the time you lose an eye. And you have no way of knowing which it will be. The same prompt, given to the same model, two days in a row, may give you a program that does what you want one time and a program that looks like it does what you want but silently corrupts your data the next time.
That’s not empowering users, that’s removing agency from users. Tools that empower users are ones that make it easy for users to build a (nicely abstracted, ignoring details that are irrelevant to them) mental model of how the system works and therefor the ability to change it in precise ways. Tools that remove agency from users take their ability to reason about how systems work and how to effect precise change.
I have zero interest in enabling tools that remove agency from users.
@david_chisnall @mjg59 I suspect CHERI would make running LLM-generated code more feasible, and probably less risky. I'm not saying this to be an annoying contrarian, but rather that stronger underlying models seems to make playing with garbage LLM code more viable. Terry Tao has been using them to generate quick and dirty proofs, cha bu duo.
-
@troed @chris_evelyn @mjg59 last time I checked, Mistral models were merely open weight, with no training dataset available nor training pipeline released as FOSS. Has that changed?
@zacchiro I understood the ask I replied to was regarding ethical training. Mistral, as an EU company, has to abide by EU regulations AI companies in the US, China etc don't have to.
-
@jenesuispasgoth I mean kind of the point of free software is that people get to modify it to their own ends and that doesn't mean it has to be good - when I first started hacking things to meet my needs I was definitely writing stuff that couldn't be upstreamed, but it worked for me, and making it easier for others to do that is a win
@mjg59 @jenesuispasgoth
There are people that analyse, design and then implement as code. Those are programmers. LLM can't replace that,
If you only ever tweak someone else's design, you may not have learned to program, only learned a language, or framework or library APIs. So maybe an LLM might help, because it's a plagiarism machine. It ignores licences and the companies building them (so called "training" = copying) have violated IP, copyright, copyleft/GPL etc on a massive scale. Theft. -
I’ve heard this argument before and I disagree with it. My goal for Free Software is to enable users, but that requires users have agency. Users being able to modify code to do what they want? Great! Users being given a black box that will modify their code in a way that might do what they want but will fail in unpredictable ways, without giving them any mechanism to build a mental model of those failure modes? Terrible!
I am not a carpenter but I have an electric screwdriver. It’s great. It lets me turn screws with much less effort than a manual one. There are a bunch of places where it doesn’t work, but that’s fine, I can understand those and use the harder-to-use tool in places where it won’t work. I can build a mental model of when not to use it and why it doesn’t work and how it will fail. I love building the software equivalent of this, things that let end users change code in ways I didn’t anticipate.
But LLM coding is not like this. It’s like a nail gun that has a 1% chance of firing backwards. 99% of the time, it’s much easier than using a hammer. 1% of the time you lose an eye. And you have no way of knowing which it will be. The same prompt, given to the same model, two days in a row, may give you a program that does what you want one time and a program that looks like it does what you want but silently corrupts your data the next time.
That’s not empowering users, that’s removing agency from users. Tools that empower users are ones that make it easy for users to build a (nicely abstracted, ignoring details that are irrelevant to them) mental model of how the system works and therefor the ability to change it in precise ways. Tools that remove agency from users take their ability to reason about how systems work and how to effect precise change.
I have zero interest in enabling tools that remove agency from users.
@david_chisnall @mjg59
If the LLM industry is an assault on knowledge systems (I agree with @atax1a on this) the same goes for the proliferation of smartphones without physical keyboards.
Touch keyboards are fairly okay in landscape mode, at the expense of covering half of the screen.
No wonder people just take pictures of the lectures. -
@jenesuispasgoth @mjg59 This is not AI endorsement, but given a sufficiently large problem / codebase, I would wager you wouldn't get a reliably identical result from having a human write code for the same problem twice either.
We expect determinism from LLMs because "its computers", not because its necessary for good results.@ck right. I was thinking more in terms of, within the same prompt requesting code generation, identical needs which should generate a single function called at different sites become several functions with slight variations (sometimes only in the function name) being generated, which happens far less often in large codebases tackling the same issue.
-
@p If you're doing something other than
var++
then you're doing something wrong. Code is instructions to a machine. The description of what that code does may be creative, if the actual implementation is then you are almost certainly in a bad place.
-
@mjg59 @jenesuispasgoth
There are people that analyse, design and then implement as code. Those are programmers. LLM can't replace that,
If you only ever tweak someone else's design, you may not have learned to program, only learned a language, or framework or library APIs. So maybe an LLM might help, because it's a plagiarism machine. It ignores licences and the companies building them (so called "training" = copying) have violated IP, copyright, copyleft/GPL etc on a massive scale. Theft.@raymaccarthy I agree with the sentiment wrt IP theft (at least, insofar as, since proprietary code requires us to respect its license, the bare minimum would be to respect FLOSS licenses). If you take ethical concerns (including ecological ones) into consideration, I think there is no conversation left whatsoever.
I took @mjg59's comments from a more technical point of view, no so much ethical in that sense.
-
@raymaccarthy I agree with the sentiment wrt IP theft (at least, insofar as, since proprietary code requires us to respect its license, the bare minimum would be to respect FLOSS licenses). If you take ethical concerns (including ecological ones) into consideration, I think there is no conversation left whatsoever.
I took @mjg59's comments from a more technical point of view, no so much ethical in that sense.
@jenesuispasgoth @mjg59
Some people think they can recycle FOSS from one licence to another using LLM, such as GPL2 to MIT or whatever. They are IP thieves.
All FOSS code, any so called copyleft licence, is actually copyright. Public domain code is a special case and in reality rare for anything written in the last 50 years. All of AT&T UNIX is still copyright.
Even programs or OS where the source has been made public with limitation for use is mostly still some sort of copyright. -
@mjg59 Yeah, as soon as there‘s an ethically sourced and trained free LLM that‘s not controlled by very shitty companies I‘m totally on board with you.
Until then we shouldn’t let that shit near our projects.
@chris_evelyn @mjg59 … and that doesn't boost global warming and slurp up much needed water in order to train and run ...
-
Free software people: A major goal of free software is for individuals to be able to cause software to behave in the way they want it to
LLMs: (enable that)
Free software people: Oh no not like that@mjg59 you mean "not by paying monthly $200 to a wanna be megacorp"? Yeah, not like that indeed.
13 years old me started coding on an old Windows 3.1 workstation with ~$0 monthly cost. If I were to enter the industry now, when one has to invest in LLMs, which btw also prevent from gaining actual skills and erode existing skills, I would simply have not done that. Must be why genZ hates LLMs
I don't see how one can look at the thought-extruding machine and think "surely it will liberate me"
-
Clearly my most unpopular thread ever, so let me add a clarification: submitting LLM generated code you don't understand to an upstream project is absolute bullshit and you should never do that. Having an LLM turn an existing codebase into something that meets your local needs? Do it. The code may be awful, it may break stuff you don't care about, and that's what all my early patches to free software looked like. It's ok to solve your problem locally.
@mjg59 pretty much. I deeply dislike any PRs I see on various projects where the prompt was basically just something like "I want you to implement this major feature into this project", with no real understanding of the underlying code and whatnot.
I would rather have coders that know what they're doing and that understand their codebases use LLMs than a random Joe Schmoe like those TikTok vibecoders with like 5 monitor screens, brainrotted on short-form content asking Claude to add E2EE to some project or to refactor the rendering process of a game engine or whatnot.
These people are wasting the maintainers' time with a jumbled mess of AI code that assumes a few things and that likely breaks on the first try.
---
There's nothing wrong with pulling a git repo and then vibe-coding a quick thing as a test or for your specific use case, but there's everything wrong with upstreaming that as a PR if you have no idea how the project's code even works or how it's architected, and with no tests or checks.
-
Personally I'm not going to literally copy code from a codebase under an incompatible license because that is what the law says, but have I read proprietary code and learned the underlying creative aspect and then written new code that embodies it? Yes! Anyone claiming otherwise is lying!
@mjg59 This might be the dumbest thing you have written. You basically just said anyone who claims not to have committed copyright infringement is lying, which is both obviously false and insulting to developers.
