Claude code source "leaks" in a mapfile
-
If i can slip in a quick PSA while my typically sleepy notifications are exploding, these are all very annoying things to say and you might want to reconsider whether they're worth ever saying in a reply directed at someone else - who are they for? what do they add?
- "why are you surprised"/"even worse than
{thing}itself is people being surprised at{thing}": unless the person is saying "i am surprised by this" they are likely not surprised by the thing. just saying something doesn't mean you are surprised by it, and people talking about something usually have paid attention to it before the moment you are encountering them. this is pointless hostility to people who are saying something you supposedly agree with so much that you think everyone should already believe it - "it's always been like this": slightly different than above. unless someone is saying "this is literally new and nothing like this has happened before" or you are adding actual historical context that you know for sure they don't already know, you're basically saying "hey did you know this thing you care enough about to be paying attention to and talking about frequently has happened before now as well." this is so easy to frame in a way that says "yes and" rather than "i assume you dont know about the things i know about due to being very smart." eg. "dang not again, they keep doing
{thing}" - "
{thing}might be bad, but{alternative/unrelated, unmentioned, non-mutually exclusive thing}is even worse": multiple things can be bad at the same time and not mentioning something does not mean i don't think it's also bad - "funny how people who think
{thing}is bad also think{alternative/unrelated, unmentioned thing}is good": closely related to the above, just because you have binarized your thinking does not mean everyone else has.
anyway if the mental image you are conjuring for your interlocuters positions them as always knowing less than you by default, that might be something to look into in yourself!
i sort of love how LLM comments sometimes tell entire stories that nobody asked. claude code even has specific system prompt language for this, but they always end up making comments about what something used to do like "now we do x instead of y" like... ok? that is why i am reading current version of code!
so claude code is just not capable of rescuing itself from its own context - if an entry in its context window throws an error, it just keep throwing that error forever until you clear it. good stuff.
(and, of course we read the entire file before checking this, rather than just reading the first 5 bytes)
- "why are you surprised"/"even worse than
-
i sort of love how LLM comments sometimes tell entire stories that nobody asked. claude code even has specific system prompt language for this, but they always end up making comments about what something used to do like "now we do x instead of y" like... ok? that is why i am reading current version of code!
so claude code is just not capable of rescuing itself from its own context - if an entry in its context window throws an error, it just keep throwing that error forever until you clear it. good stuff.
(and, of course we read the entire file before checking this, rather than just reading the first 5 bytes)
this is super minor, and i've seen this in human code plenty of times, but this is the norm of this app verging on being formal code style.
so you have a file reading tool, you need to declare what kinds of file extensions it supports. that's very normal. claude code takes the interesting strategy of defining what extensions it doesn't read. that's also defensible, there are a zillion text extensions. i've seen strategies that just read an initial range of bytes and see if some proportion of them are ascii or unicode.
where does this get declared? why of course in as many places as there are rules.
hasBinaryExtension()comes fromconstants/files.ts,isPDFExtension()comes fromutils/pdfUtils.ts(which checks if the file extension is a member of the set{'pdf'}), andIMAGE_EXTENSIONSis declared in theFileReadTool.tsfile.of course, elsewhere we also have
IMAGE_EXTENSION_REGEXfromutils/imagePaste(sometimes used directly, other times with its wrapperisImageFilePath),TEXT_FILE_EXTENSIONSinutils/claudemd.ts. and we also have many inlined mime type lists and sets. and all of these somehow manage to implement the check differently. so rather than having, for example, agetFileType()function, we have both exactly the same and kinda the same logic redone in place every time it is done, which is hundreds of times. but that's none of my business, that's just how code works now and i need to get with the times.
-
this is super minor, and i've seen this in human code plenty of times, but this is the norm of this app verging on being formal code style.
so you have a file reading tool, you need to declare what kinds of file extensions it supports. that's very normal. claude code takes the interesting strategy of defining what extensions it doesn't read. that's also defensible, there are a zillion text extensions. i've seen strategies that just read an initial range of bytes and see if some proportion of them are ascii or unicode.
where does this get declared? why of course in as many places as there are rules.
hasBinaryExtension()comes fromconstants/files.ts,isPDFExtension()comes fromutils/pdfUtils.ts(which checks if the file extension is a member of the set{'pdf'}), andIMAGE_EXTENSIONSis declared in theFileReadTool.tsfile.of course, elsewhere we also have
IMAGE_EXTENSION_REGEXfromutils/imagePaste(sometimes used directly, other times with its wrapperisImageFilePath),TEXT_FILE_EXTENSIONSinutils/claudemd.ts. and we also have many inlined mime type lists and sets. and all of these somehow manage to implement the check differently. so rather than having, for example, agetFileType()function, we have both exactly the same and kinda the same logic redone in place every time it is done, which is hundreds of times. but that's none of my business, that's just how code works now and i need to get with the times.i love this. there's a mechanism to slip secret messages to the LLM that it is told to interpret as system messages. there is no validation around these of any kind on the client, and there doesn't seem to be any differentiation about location or where these things happen, so that seems like a nice prompt injection vector. this is how claude code reminds the LLM to not do a malware, and it's applied by just string concatenation. i can't find any place that gets stripped aside from when displaying output. it actually looks like all the system reminders get catted together before being send to the API. neat!
-
i love this. there's a mechanism to slip secret messages to the LLM that it is told to interpret as system messages. there is no validation around these of any kind on the client, and there doesn't seem to be any differentiation about location or where these things happen, so that seems like a nice prompt injection vector. this is how claude code reminds the LLM to not do a malware, and it's applied by just string concatenation. i can't find any place that gets stripped aside from when displaying output. it actually looks like all the system reminders get catted together before being send to the API. neat!
continuing thoughts in: https://neuromatch.social/@jonny/116328409651740378
one thing that is clear from reading a lot of LLM code - and this is obvious from the nature of the models and their application - is that it is big on the form of what it loves to call "architecture" even if in toto it makes no fucking sense.
So here you have some accessor function
isPDFExtensionthat checks if some string is a member of the setDOCUMENT_EXTENSIONS(which is a constant with a single member "pdf"). That is an extremely reasonable pattern: you have a bunch of disjoint sets of different kinds of extensions - binary extensions, image extensions, etc. and then you can do set operations like unions and differences and intersections and whatnot to create a bunch of derived functions that can handle dynamic operations that you couldn't do well with a bunch of consts. then just make the functional form the standard calling pattern (and even make a top-level wrapper likegetFileType) and you have the oft fabled "abstraction." that's a reasonable ass system that provides a stable calling surface and a stable declaration surface. hell it would probably even help the LLM code if it was already in place because it's a predictable rules-based system.but what the LLMs do is in one narrow slice of time implement the "is member of set
{pdf}" version robustly one time, and then they implement the regex pattern version flexibly another time, and then they implement theany str.endswith()version modularly another time, and so on. Of course usually in-place, and different file naming patterns are part of the architecture when it's feeling a little too spicy to stay in place.This is an important feature of the gambling addiction formulation of these tools: only the margin matters, the last generation. it carefully regulates what it shows you to create a space of potential reward and closes the gap. It's episodic TV, gameshows for code: someone wins every week, but we get cycles in cycles of seeming progression that always leave one stone conspicuously unturned. The intermediate comments from the LLM where it discovers prior structure and boldly decides to forge ahead brand new are also part of the reward cycle: we are going up, forever. cleaning up after ourselves is down there.
Tech debt is when you have banked a lot of story hours and are finally due for a big cathartic shift and set the LLM loose for "the big cleanup." this is also very similar to the tools that scam mobile games use (for those who don't know me, i spent roughly six months with daily scheduled (carefully titrated lmao) time playing the worst scam mobile chum games i could find to try and experience what the grip of that addition is like without uh losing a bunch of money).
Unlike slot machines or table games, which have a story horizon limited by how long you can sit in the same place, mobile games can establish a space of play that's broader and more continuous. so they always combine several shepherd's tone reward ladders at once - you have hit the session-length intermittent reward cap in the
arenamodality which gets youcoins, so you need to go "recharge" by playing theversusmodality which gets yougems. (Typically these are also mixed - one modality gets you some proportion of resourcex, y, z, another gets you a different proportion, and those are usually unstable).Of course it doesn't fucking matter what the modality is. they are all the same. in the scam mobile games sometimes this is literally the case, where if you decompile them, they have different menu wrappings that all direct into the same scene. you're still playing the game, that's all that matters. The goal of the game design is to chain together several time cycles so that you can win->lose in one, win->lose in another... and then by the time you have made the rounds you come back to the first and you are refreshed and it's new. So you have momentary mana wheels, daily earnings caps, weekly competitions, seasonal storylines, and all-time leaderboards.
That's exactly the cycle that programming with LLMs tap into. You have momentary issues, and daily project boards, and weekly sprints, and all-time star counts, and so on. Accumulate tech debt by new features, release that with "cleanup," transition to "security audit." Each is actually the same, but the present themselves as the continuation of and solution to the others. That overlaps with the token limitations, and the claude code source is actually littered with lots of helpful panic nudges for letting you know that you're reaching another threshold. The difference is that in true gambling the limit is purely artificial - the coins are an integer in some database. with LLMs the limitation is physical - compute costs fucking money baby. but so is the reward. it's the same in the game, and the whales come around one way or another.
A series of flashing lights and pictures, set membership, regex, green checks, the feeling of going very fast but never making it anywhere. except in code you do make it somewhere, it's just that the horizon falls away behind you and the places you were before disappear. and sooner or later only anthropic can really afford to keep the agents running 24/7 tending to the slop heap - the house always wins.
-
continuing thoughts in: https://neuromatch.social/@jonny/116328409651740378
one thing that is clear from reading a lot of LLM code - and this is obvious from the nature of the models and their application - is that it is big on the form of what it loves to call "architecture" even if in toto it makes no fucking sense.
So here you have some accessor function
isPDFExtensionthat checks if some string is a member of the setDOCUMENT_EXTENSIONS(which is a constant with a single member "pdf"). That is an extremely reasonable pattern: you have a bunch of disjoint sets of different kinds of extensions - binary extensions, image extensions, etc. and then you can do set operations like unions and differences and intersections and whatnot to create a bunch of derived functions that can handle dynamic operations that you couldn't do well with a bunch of consts. then just make the functional form the standard calling pattern (and even make a top-level wrapper likegetFileType) and you have the oft fabled "abstraction." that's a reasonable ass system that provides a stable calling surface and a stable declaration surface. hell it would probably even help the LLM code if it was already in place because it's a predictable rules-based system.but what the LLMs do is in one narrow slice of time implement the "is member of set
{pdf}" version robustly one time, and then they implement the regex pattern version flexibly another time, and then they implement theany str.endswith()version modularly another time, and so on. Of course usually in-place, and different file naming patterns are part of the architecture when it's feeling a little too spicy to stay in place.This is an important feature of the gambling addiction formulation of these tools: only the margin matters, the last generation. it carefully regulates what it shows you to create a space of potential reward and closes the gap. It's episodic TV, gameshows for code: someone wins every week, but we get cycles in cycles of seeming progression that always leave one stone conspicuously unturned. The intermediate comments from the LLM where it discovers prior structure and boldly decides to forge ahead brand new are also part of the reward cycle: we are going up, forever. cleaning up after ourselves is down there.
Tech debt is when you have banked a lot of story hours and are finally due for a big cathartic shift and set the LLM loose for "the big cleanup." this is also very similar to the tools that scam mobile games use (for those who don't know me, i spent roughly six months with daily scheduled (carefully titrated lmao) time playing the worst scam mobile chum games i could find to try and experience what the grip of that addition is like without uh losing a bunch of money).
Unlike slot machines or table games, which have a story horizon limited by how long you can sit in the same place, mobile games can establish a space of play that's broader and more continuous. so they always combine several shepherd's tone reward ladders at once - you have hit the session-length intermittent reward cap in the
arenamodality which gets youcoins, so you need to go "recharge" by playing theversusmodality which gets yougems. (Typically these are also mixed - one modality gets you some proportion of resourcex, y, z, another gets you a different proportion, and those are usually unstable).Of course it doesn't fucking matter what the modality is. they are all the same. in the scam mobile games sometimes this is literally the case, where if you decompile them, they have different menu wrappings that all direct into the same scene. you're still playing the game, that's all that matters. The goal of the game design is to chain together several time cycles so that you can win->lose in one, win->lose in another... and then by the time you have made the rounds you come back to the first and you are refreshed and it's new. So you have momentary mana wheels, daily earnings caps, weekly competitions, seasonal storylines, and all-time leaderboards.
That's exactly the cycle that programming with LLMs tap into. You have momentary issues, and daily project boards, and weekly sprints, and all-time star counts, and so on. Accumulate tech debt by new features, release that with "cleanup," transition to "security audit." Each is actually the same, but the present themselves as the continuation of and solution to the others. That overlaps with the token limitations, and the claude code source is actually littered with lots of helpful panic nudges for letting you know that you're reaching another threshold. The difference is that in true gambling the limit is purely artificial - the coins are an integer in some database. with LLMs the limitation is physical - compute costs fucking money baby. but so is the reward. it's the same in the game, and the whales come around one way or another.
A series of flashing lights and pictures, set membership, regex, green checks, the feeling of going very fast but never making it anywhere. except in code you do make it somewhere, it's just that the horizon falls away behind you and the places you were before disappear. and sooner or later only anthropic can really afford to keep the agents running 24/7 tending to the slop heap - the house always wins.
If you are reading an image and near your estimated token limit, first try to
compressImageBufferWithTokenLimit, then if that fails with any kind of error, try and usesharpdirectly and resize it to 400x400, cropping. finally, fuck it, just throw the buffer at the API.of course
compressImageBufferWithTokenLimitis also compression withsharp, and is also a series of fallback operations. We start by trying to detect the image encoding that we so painstakingly learned from... the file extension... but if we can't fuck it that shit is a jpeg now.then, even if it's fine and we don't need to do anything, we still re-compress it (wait, no even though it's named createCompressedImageResult, it does nothing). Otherwise, we yolo our way through another layer of fallbacks, progressive resizing, palletized PNGs, back to JPEG again, and then on to "ultra compressed JPEG" which is... incredibly... exactly the same as the top-level in-place code in the parent function
while two of the legs return a
createImageReponse, the first leg returns acompressedImageResponsebut then unpacks that back into an object literal that's almost exactly the same except we call ittypeinstead ofmediaType. -
If you are reading an image and near your estimated token limit, first try to
compressImageBufferWithTokenLimit, then if that fails with any kind of error, try and usesharpdirectly and resize it to 400x400, cropping. finally, fuck it, just throw the buffer at the API.of course
compressImageBufferWithTokenLimitis also compression withsharp, and is also a series of fallback operations. We start by trying to detect the image encoding that we so painstakingly learned from... the file extension... but if we can't fuck it that shit is a jpeg now.then, even if it's fine and we don't need to do anything, we still re-compress it (wait, no even though it's named createCompressedImageResult, it does nothing). Otherwise, we yolo our way through another layer of fallbacks, progressive resizing, palletized PNGs, back to JPEG again, and then on to "ultra compressed JPEG" which is... incredibly... exactly the same as the top-level in-place code in the parent function
while two of the legs return a
createImageReponse, the first leg returns acompressedImageResponsebut then unpacks that back into an object literal that's almost exactly the same except we call ittypeinstead ofmediaType.for those keeping score at home, we have the opportunity to re-compress the same image nine times
-
for those keeping score at home, we have the opportunity to re-compress the same image nine times
holy shit there's another entire fallback tree before this one, that's actually an astounding twenty two times it's possible to compress an image across nine independent conditional legs of code in a single api call. i can't even screenshot this, the spaghetti is too powerful
-
holy shit there's another entire fallback tree before this one, that's actually an astounding twenty two times it's possible to compress an image across nine independent conditional legs of code in a single api call. i can't even screenshot this, the spaghetti is too powerful
here, if i fold all the
returnblocks and decrease my font size as small as it goes i can fit all the compression invocations in the first of three top-level compression fallback trees in a single screenshot, but since it is so small i just have to circle them in red like it's a football diagram.this function is named "maybeResizeAndDownsampleImageBuffer" and boy that is a hell of a maybe!
-
here, if i fold all the
returnblocks and decrease my font size as small as it goes i can fit all the compression invocations in the first of three top-level compression fallback trees in a single screenshot, but since it is so small i just have to circle them in red like it's a football diagram.this function is named "maybeResizeAndDownsampleImageBuffer" and boy that is a hell of a maybe!
and what if i told you that if it passes a page range to its pdf reader, it first extracts those pages to separate images and then calls this function in a loop on each of the pages. so you have the privilege of compressing
n_pagesimagesn_pages * 13times.this function is used 13 times: in the file reader, in the mcp result handler, in the bash tool, and in the clipboard handler - each of which has their entire own surrounding image handling routines that are each hundreds of lines of similar but still very different fallback code to do exactly the same thing.
so that's where all the five hundred thousand lines come from - fallback conditions and then more fallback conditions to compensate for the variable output of all the other fallback conditions. thirteen butts pooping, back and forth, forever.
-
and what if i told you that if it passes a page range to its pdf reader, it first extracts those pages to separate images and then calls this function in a loop on each of the pages. so you have the privilege of compressing
n_pagesimagesn_pages * 13times.this function is used 13 times: in the file reader, in the mcp result handler, in the bash tool, and in the clipboard handler - each of which has their entire own surrounding image handling routines that are each hundreds of lines of similar but still very different fallback code to do exactly the same thing.
so that's where all the five hundred thousand lines come from - fallback conditions and then more fallback conditions to compensate for the variable output of all the other fallback conditions. thirteen butts pooping, back and forth, forever.
there is a callback feature "file read listeners" which is only called if the file type is a text document, gated for anthropic employees only, such that whenever a text file is read (any part of any text file, which often happens in a rapid series with subranges when it does 'explore' mode, rather than just like grepping), another subagent running sonnet is spun off to update a "magic doc" markdown file that summarizes the file that's read - that's one "magic doc" per file, not one magic doc.
I have yet to get into the tool/agent graph situation in earnest, but keep in mind that this is an entirely single-use and completely different means of spawning a graph of subagents off a given tool call than is used anywhere else.
Spoiler alert for what i'm gonna check out next is that claude code has no fucking tool calling execution model it just calls whatever the fuck it wants wherever the fuck it wants. Tools are or less a convenient fiction. I have only read one completely (file read) and skimmed a dozen more but they essentially share nothing in common except for a humongous list of often-single-use params and the return type of "any object with a single key and whatever else"
i'm in hell. this is hell.
-
there is a callback feature "file read listeners" which is only called if the file type is a text document, gated for anthropic employees only, such that whenever a text file is read (any part of any text file, which often happens in a rapid series with subranges when it does 'explore' mode, rather than just like grepping), another subagent running sonnet is spun off to update a "magic doc" markdown file that summarizes the file that's read - that's one "magic doc" per file, not one magic doc.
I have yet to get into the tool/agent graph situation in earnest, but keep in mind that this is an entirely single-use and completely different means of spawning a graph of subagents off a given tool call than is used anywhere else.
Spoiler alert for what i'm gonna check out next is that claude code has no fucking tool calling execution model it just calls whatever the fuck it wants wherever the fuck it wants. Tools are or less a convenient fiction. I have only read one completely (file read) and skimmed a dozen more but they essentially share nothing in common except for a humongous list of often-single-use params and the return type of "any object with a single key and whatever else"
i'm in hell. this is hell.
i have been writing a graph processing library for about a year now and if i was a fucking AI grifter here is where i would plug it as like "actually a graph processor library" and "could do all of what claude code does without fucking being the worst nightmare on ice money can buy."
I say that not as self promo, but as a way of saying how in the FUCK do you FUCK UP graph processing this badly. these people make like tens of times more money than i do but their work is just tamping down a volley of dessicated backpacking poops into muskets and then free firing it into the fucking economy
-
i have been writing a graph processing library for about a year now and if i was a fucking AI grifter here is where i would plug it as like "actually a graph processor library" and "could do all of what claude code does without fucking being the worst nightmare on ice money can buy."
I say that not as self promo, but as a way of saying how in the FUCK do you FUCK UP graph processing this badly. these people make like tens of times more money than i do but their work is just tamping down a volley of dessicated backpacking poops into muskets and then free firing it into the fucking economy
you can TELL that this technology REALLY WORKS by how the people that made it and presumably know how to use it the best out of everyone CANT EVEN USE IT TO EDIT A FUCKING FILE RELIABLY and have to resort to multiple stern allcaps reminders to the robot that "you must not change the fucking header metadata you scoundrel" which for the rest of ALL OF COMPUTING is not even an afterthought because literally all it requires is "split the first line off and don't change that one" because ALL OF THE REST OF COMPUTING can make use of the power of INTEGERS.
-
you can TELL that this technology REALLY WORKS by how the people that made it and presumably know how to use it the best out of everyone CANT EVEN USE IT TO EDIT A FUCKING FILE RELIABLY and have to resort to multiple stern allcaps reminders to the robot that "you must not change the fucking header metadata you scoundrel" which for the rest of ALL OF COMPUTING is not even an afterthought because literally all it requires is "split the first line off and don't change that one" because ALL OF THE REST OF COMPUTING can make use of the power of INTEGERS.
alrighty so that's one of 43 tools read, the tools directory being 38494 source lines out of 390592 source lines, 513221 total lines. I need to go to bed. This is the most fabulously, flamboyantly bad code i have ever encountered.
Worth noting I was reading the file reading tool because i thought it would be the simplest possible thing one could do because it basically shouldn't be doing anything except preparing and sending strings or bytes to the backend.
I expected to get some sense of "ok what is the format of the data as it's passed around within the program, surely text strings are a basic unit of currency. No dice. Fewer than no dice. Negative dice somehow.
-
alrighty so that's one of 43 tools read, the tools directory being 38494 source lines out of 390592 source lines, 513221 total lines. I need to go to bed. This is the most fabulously, flamboyantly bad code i have ever encountered.
Worth noting I was reading the file reading tool because i thought it would be the simplest possible thing one could do because it basically shouldn't be doing anything except preparing and sending strings or bytes to the backend.
I expected to get some sense of "ok what is the format of the data as it's passed around within the program, surely text strings are a basic unit of currency. No dice. Fewer than no dice. Negative dice somehow.
next puzzle: why in the fuck are some of the tools actually two tools for entering and exiting being in the tool state. none of the other tools are like that. one is simply in the tool state by calling the tool. Plan mode is also an agent. Plan Agent. and Agent is also a tool. Agent Tool. Tools can be agents and agents can be tools. Tools can spawn agents (but they don't need to call the agent tool) and agents can call tools (however there is no tool agent). What is going on. What is anything.
-
next puzzle: why in the fuck are some of the tools actually two tools for entering and exiting being in the tool state. none of the other tools are like that. one is simply in the tool state by calling the tool. Plan mode is also an agent. Plan Agent. and Agent is also a tool. Agent Tool. Tools can be agents and agents can be tools. Tools can spawn agents (but they don't need to call the agent tool) and agents can call tools (however there is no tool agent). What is going on. What is anything.
"the emperor is not only naked, he's smooth like a ken doll down there and i'm pretty sure that's just a mannequin with a colony of rats living inside it anyway"
-
"the emperor is not only naked, he's smooth like a ken doll down there and i'm pretty sure that's just a mannequin with a colony of rats living inside it anyway"
I seriously need to work on my actual job today but i am giving myself 15 minutes to peek at the agent tool prompts as a treat.
"regulations are written in blood" seems like too dramatic of a way to phrase it, but these system prompts are very revealing about the intrinsically busted nature of using these tools for anything deterministic (read: anything you actually want to happen). Each guard in the prompt presumably refers to something that has happened before, but also, since the prompts actually don't work to prevent the thing they are describing, they are also documentation of bugs that are almost certain to happen again. Many of the prompt guards form pairs with attempted code mitigations (or, they would be pairs if the code was written with any amount of sense, it's really like... polycules...), so they are useful to guide what kind of fucked up shit you should be looking for.
so this is part of the prompt for the "agent tool" that launches forked agents (that receive the parent context, "subagents" don't). The purpose of the forked agent is to do some additional tool calls and get some summary for a small subproblem within the main context. Apparently it is difficult to make this actually happen though, as the parent LLM likes to launch the forked agent and just hallucinate a response as if the forked agent had already completed.
-
I seriously need to work on my actual job today but i am giving myself 15 minutes to peek at the agent tool prompts as a treat.
"regulations are written in blood" seems like too dramatic of a way to phrase it, but these system prompts are very revealing about the intrinsically busted nature of using these tools for anything deterministic (read: anything you actually want to happen). Each guard in the prompt presumably refers to something that has happened before, but also, since the prompts actually don't work to prevent the thing they are describing, they are also documentation of bugs that are almost certain to happen again. Many of the prompt guards form pairs with attempted code mitigations (or, they would be pairs if the code was written with any amount of sense, it's really like... polycules...), so they are useful to guide what kind of fucked up shit you should be looking for.
so this is part of the prompt for the "agent tool" that launches forked agents (that receive the parent context, "subagents" don't). The purpose of the forked agent is to do some additional tool calls and get some summary for a small subproblem within the main context. Apparently it is difficult to make this actually happen though, as the parent LLM likes to launch the forked agent and just hallucinate a response as if the forked agent had already completed.
The prompt strings have an odd narrative/narrator structure. It sort of reminds me of Bakhtin's discussion of polyphony and narrator in Dostoevsky - there is no omniscient narrator, no author-constructed reality. narration is always embedded within the voice and subjectivity of the character. this is also literally true since the LLM is writing the code and the prompts that are then used to write code and prompts at runtime.
They also read a bit like a Philip K Dick story, paranoid and suspicious, constantly uncertain about the status of one's own and others identities.
-
The prompt strings have an odd narrative/narrator structure. It sort of reminds me of Bakhtin's discussion of polyphony and narrator in Dostoevsky - there is no omniscient narrator, no author-constructed reality. narration is always embedded within the voice and subjectivity of the character. this is also literally true since the LLM is writing the code and the prompts that are then used to write code and prompts at runtime.
They also read a bit like a Philip K Dick story, paranoid and suspicious, constantly uncertain about the status of one's own and others identities.
oh. hm. that seems bad. "workers aren't affected by the parent's tool restrictions."
It's hard to tell what's going on here because claude code doesn't really use typescript well - many of the most important types are dynamically computed from
any, and most of the time when types do exist many of their fields are nullable and the calling code has elaborate fallback conditions to compensate. all of which sort of defeats the purpose of ts.So i need to trace out like a dozen steps to see how the permission mode gets populated. But this comment is... concerning...
-
oh. hm. that seems bad. "workers aren't affected by the parent's tool restrictions."
It's hard to tell what's going on here because claude code doesn't really use typescript well - many of the most important types are dynamically computed from
any, and most of the time when types do exist many of their fields are nullable and the calling code has elaborate fallback conditions to compensate. all of which sort of defeats the purpose of ts.So i need to trace out like a dozen steps to see how the permission mode gets populated. But this comment is... concerning...
ok over my 15 minute allotment by an hour. brb
-
I seriously need to work on my actual job today but i am giving myself 15 minutes to peek at the agent tool prompts as a treat.
"regulations are written in blood" seems like too dramatic of a way to phrase it, but these system prompts are very revealing about the intrinsically busted nature of using these tools for anything deterministic (read: anything you actually want to happen). Each guard in the prompt presumably refers to something that has happened before, but also, since the prompts actually don't work to prevent the thing they are describing, they are also documentation of bugs that are almost certain to happen again. Many of the prompt guards form pairs with attempted code mitigations (or, they would be pairs if the code was written with any amount of sense, it's really like... polycules...), so they are useful to guide what kind of fucked up shit you should be looking for.
so this is part of the prompt for the "agent tool" that launches forked agents (that receive the parent context, "subagents" don't). The purpose of the forked agent is to do some additional tool calls and get some summary for a small subproblem within the main context. Apparently it is difficult to make this actually happen though, as the parent LLM likes to launch the forked agent and just hallucinate a response as if the forked agent had already completed.
@jonny if someone were to take seriously the task of archecting this, you’d want a framework that doesn’t use prompts for this right? something that treats the LLM output more like untrusted stochastic guesses at solutions, where these prompt rules are written as a test instead of a prompt
