[jacking off motion] great 🙄
-
@SnoopJ it really feels… gross, doesn’t it? corrupting? like a warm blanket for the mind that reeks of mildew. this is pure aesthetics I know but the more interactions I have with it the worse it gets
@glyph the little death that brings obliteration, perhaps
-
@SnoopJ it really feels… gross, doesn’t it? corrupting? like a warm blanket for the mind that reeks of mildew. this is pure aesthetics I know but the more interactions I have with it the worse it gets
@glyph @SnoopJ I've mostly abstained from trying these tools in the process of criticism on the basis of ethical concerns, but secondarily — I know that the makers of these products are trying to trick me into thinking that they're something they're not. I don't trust my own cognition enough to keep putting it into danger, I guess?
-
@glyph @SnoopJ I've mostly abstained from trying these tools in the process of criticism on the basis of ethical concerns, but secondarily — I know that the makers of these products are trying to trick me into thinking that they're something they're not. I don't trust my own cognition enough to keep putting it into danger, I guess?
-
@glyph the little death that brings obliteration, perhaps
@glyph I can see how people, even experts, getting themselves into the scenario of "I have absolutely no idea what this code is doing or how it is organized and the only way out is by using the model harder"
(of course, the example of this most immediately visible to me should be a slam dunk: "oh jeez I forgot to tell it to write tests from the start" → "okay why can't you do that now?" → [anxiety noises])
-
@glyph @SnoopJ I don't think I'm an especial outlier on addiction susceptibility, in either direction. At most, I think I can say that most propaganda is designed for people who think fairly differently than I do... if someone is aiming more specifically at people like me, I don't think I'm immune.
I've seen far too many people who are at least vaguely similar to me come completely unglued on contact with these things, and I don't trust that I can resist that.
-
@glyph @SnoopJ I've mostly abstained from trying these tools in the process of criticism on the basis of ethical concerns, but secondarily — I know that the makers of these products are trying to trick me into thinking that they're something they're not. I don't trust my own cognition enough to keep putting it into danger, I guess?
@xgranade @glyph yea, I had some thoughts about how bad this probably is for *my neurotype* (wtfever that is) specifically.
I am trying to "set those things aside" (ugh, those words taste bad) and approach this from a steel-man perspective, and answer: is it fit for purpose at the most mercenary level?
But I had not realized until I started using it just how awful the regular pauses in my work would be. Sure, I could turn my attention to something else while the model spins, but I know where *that* road goes and it's nowhere good.
And otherwise, I'm just sitting there waiting for my next cleanup on Aisle 3, and the fatigue set in quickly.
-
@xgranade @glyph yea, I had some thoughts about how bad this probably is for *my neurotype* (wtfever that is) specifically.
I am trying to "set those things aside" (ugh, those words taste bad) and approach this from a steel-man perspective, and answer: is it fit for purpose at the most mercenary level?
But I had not realized until I started using it just how awful the regular pauses in my work would be. Sure, I could turn my attention to something else while the model spins, but I know where *that* road goes and it's nowhere good.
And otherwise, I'm just sitting there waiting for my next cleanup on Aisle 3, and the fatigue set in quickly.
@xgranade @glyph I'm almost relieved that it consumed as much compute as it did and *still* made mistakes on what was relatively still quite a straightforward task, even if it did involve some JSONSchema fancy footwork, and even if it did surface some issues.
Based on today's experimentation, the juice does not seem worth the squeeze. I have more evaluation to do in order to have my ducks in a row for when the Enthusiasts in my life come calling asking why I feel the way I do about this, but yea. Unsettling experience so far.
-
But it was like a breath of fresh air to finish with the schema work, then point `datamodel-code-generator` (a deterministic tool) at those files and generate corresponding Pydantic model code for the schema.
Not only was that Python code *much* easier to review (not the model's fault, JSONSchema is just very difficult for me to read) but it was just… so nice to be touching a tool whose behavior I can rely on. I didn't need to check its work very closely at all, because I know that it's applying a fixed set of well-characterized rules to generate those models from the (equally well-defined) schema.
@SnoopJ I like datamodel-code-generator, but did have a few utterly baffling sessions with it that turned out to be due to a missing deepcopy call: https://github.com/koxudaxi/datamodel-code-generator/pull/2215
(I shudder to think what nonsense an LLM might spew if it hit a tool bug like that)
-
@SnoopJ I like datamodel-code-generator, but did have a few utterly baffling sessions with it that turned out to be due to a missing deepcopy call: https://github.com/koxudaxi/datamodel-code-generator/pull/2215
(I shudder to think what nonsense an LLM might spew if it hit a tool bug like that)
@ancoghlan I imagine you've put it to much heavier-duty use than I am, but good to know!
I also shudder to think. I suppose I should find a suitable bug from my own recent past and see how long is the path from the initial report to proper characterization of the issue (if we get there at all)
-
Coming for your job
Yikes. Just the first line of output requires both March 3rd and March 5th to be Tuesday—at least, if you remember obscure facts like “Tuesday is the day after Monday” and “there are 7 days in a week.”
Truly this application has a dizzying intellect and can be completely trusted with any other modular arithmetic it might stumble across.
-
But it was like a breath of fresh air to finish with the schema work, then point `datamodel-code-generator` (a deterministic tool) at those files and generate corresponding Pydantic model code for the schema.
Not only was that Python code *much* easier to review (not the model's fault, JSONSchema is just very difficult for me to read) but it was just… so nice to be touching a tool whose behavior I can rely on. I didn't need to check its work very closely at all, because I know that it's applying a fixed set of well-characterized rules to generate those models from the (equally well-defined) schema.
gave the bullshit machine another gentle pitch, fed it the report of new bug and asked for an explanation
it did correctly point to where the originating flaw was (`data=…` instead of `json=…` in `requests`), and from where I hit pause and explored possible explanations for why that suddenly mattered (FastAPI 0.132 has a breaking change associated with the "wrong" `Content-Type` header)
then, already knowing what the problem was, asked for an *explanation* rather than "oh look at this code". took several cycles of incorrect confabulation including very explicit hints ("I am sure the Pydantic version has not changed." "FastAPI is not bounded above, check the release notes") to get to an explanation that could be called correct.
I don't know how to evaluate how much faster that got me through the fog-of-war that is Pydantic's absolutely terrible error reporting, but I do know that the number of potential pitfalls on the far side of that is… not something that inspires faith.
-
gave the bullshit machine another gentle pitch, fed it the report of new bug and asked for an explanation
it did correctly point to where the originating flaw was (`data=…` instead of `json=…` in `requests`), and from where I hit pause and explored possible explanations for why that suddenly mattered (FastAPI 0.132 has a breaking change associated with the "wrong" `Content-Type` header)
then, already knowing what the problem was, asked for an *explanation* rather than "oh look at this code". took several cycles of incorrect confabulation including very explicit hints ("I am sure the Pydantic version has not changed." "FastAPI is not bounded above, check the release notes") to get to an explanation that could be called correct.
I don't know how to evaluate how much faster that got me through the fog-of-war that is Pydantic's absolutely terrible error reporting, but I do know that the number of potential pitfalls on the far side of that is… not something that inspires faith.
this kind of use at least puts *some* limitation on the blast radius of the use
this experiment and another (a descriptive task mentioned up-thread) convinces me that there are still many pitfalls when using such a model to guide this kind of exploratory work
but at least in that case, the user of the tool is the one who is having their time wasted the most, and bullshit does not as easily make it into the codebase.
assuming the tool has been instructed not to generate any and the user has not directly subverted that, anyway. which is of course a big assumption, sufficient Enthusiasm will (does) clearly lead to this kind of subversion and lying about it
-
this kind of use at least puts *some* limitation on the blast radius of the use
this experiment and another (a descriptive task mentioned up-thread) convinces me that there are still many pitfalls when using such a model to guide this kind of exploratory work
but at least in that case, the user of the tool is the one who is having their time wasted the most, and bullshit does not as easily make it into the codebase.
assuming the tool has been instructed not to generate any and the user has not directly subverted that, anyway. which is of course a big assumption, sufficient Enthusiasm will (does) clearly lead to this kind of subversion and lying about it
To identify and explain the problem, and generate (but NOT apply) diffs for this issue (which can be fixed in -2+2, which I wrote before asking), the model consumed:
2,900,000 tokens
5 min 30 seconds of API time
12 "premium" requests -
To identify and explain the problem, and generate (but NOT apply) diffs for this issue (which can be fixed in -2+2, which I wrote before asking), the model consumed:
2,900,000 tokens
5 min 30 seconds of API time
12 "premium" requestsoh also when I asked for an explanatory comment to be added, I did have to instruct the model not to masquerade as a human author when populating the tags required by the project's comment convention
so that's great
-
gave the bullshit machine another gentle pitch, fed it the report of new bug and asked for an explanation
it did correctly point to where the originating flaw was (`data=…` instead of `json=…` in `requests`), and from where I hit pause and explored possible explanations for why that suddenly mattered (FastAPI 0.132 has a breaking change associated with the "wrong" `Content-Type` header)
then, already knowing what the problem was, asked for an *explanation* rather than "oh look at this code". took several cycles of incorrect confabulation including very explicit hints ("I am sure the Pydantic version has not changed." "FastAPI is not bounded above, check the release notes") to get to an explanation that could be called correct.
I don't know how to evaluate how much faster that got me through the fog-of-war that is Pydantic's absolutely terrible error reporting, but I do know that the number of potential pitfalls on the far side of that is… not something that inspires faith.
@SnoopJ Bit of a secret, not sure how open it is, but the models rely very heavily on error message quality in their little fixup loops -- if the error message doesn't basically spell it out, it's not like the model has, y'know, a theory of how to model computers
-
R relay@relay.mycrowd.ca shared this topic
-
@SnoopJ Bit of a secret, not sure how open it is, but the models rely very heavily on error message quality in their little fixup loops -- if the error message doesn't basically spell it out, it's not like the model has, y'know, a theory of how to model computers
@delta_vee it actually did fine with this part of the problem, one-shotted it more or less instantly. spotted the subtle difference between `{…}` and `'{…}'` in the report and correctly diagnosed that the JSON payload was not being treated as JSON.
But the working-backwards from there (`Content-Type` is wrong, request machinery is responsible, why did it break/why did it work before?) was not great.
Or do you think the error message quality for 'cascade' failures like this still matters that far into the "reasoning" process?
-
@delta_vee it actually did fine with this part of the problem, one-shotted it more or less instantly. spotted the subtle difference between `{…}` and `'{…}'` in the report and correctly diagnosed that the JSON payload was not being treated as JSON.
But the working-backwards from there (`Content-Type` is wrong, request machinery is responsible, why did it break/why did it work before?) was not great.
Or do you think the error message quality for 'cascade' failures like this still matters that far into the "reasoning" process?
@SnoopJ The error quality for cascade failures like this are even more important in terms of root cause
