So, more #genAI pain.
-
Claude Code itself works really well against local models if you have the hardware to run something in the 30b parameter range. And hey it's already been open sourced so when Anthropic ceases to exist the agent will still be around in some form.
@bflipp @virtuous_sloth I'm curious: What kind of hardware is that?
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray I do think there is a bubble that will pop, but I also think that useful AI is already escaping from the dev community via tooling like OpenClaw that is taking off and using lots of tokens automating tasks for people.
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray So, if it's real value that people are willing to pay for, is it a bubble? Or a business?
I guess if software developers are paying enough that investors think it's interesting, but not enough to pay the expenses, and not by a large margin, that's probably a bubble.
-
@timbray So, if it's real value that people are willing to pay for, is it a bubble? Or a business?
I guess if software developers are paying enough that investors think it's interesting, but not enough to pay the expenses, and not by a large margin, that's probably a bubble.
@evan depends how many people are willing to pay, I guess.
-
@evan depends how many people are willing to pay, I guess.
@timbray Right. If revenue is only 0.5% of expenses, or even 5%, there's no way that unit economics can be tuned to make a profit. But if it's 50%, say, I think the traditional VC idea is to keep funding it, and then make it pay well in the future. I don't know if those percentages are anywhere near accurate, btw.
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray
I do see some other clearly useful, clearly worth paying for use cases at work. We're a multilingual environment, and translation is if anything more of a killer app than coding. A large fraction of the entire workplace already uses it, and there's a clear business case for paying for it. -
@timbray Right. If revenue is only 0.5% of expenses, or even 5%, there's no way that unit economics can be tuned to make a profit. But if it's 50%, say, I think the traditional VC idea is to keep funding it, and then make it pay well in the future. I don't know if those percentages are anywhere near accurate, btw.
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray The bubble is all business. Investors will shutdown successful companies that make a nice 3 million a year because they decided that is the market and there is no growth.
Even in software the rate of adoption is deceptive since there are mandates on using it coming from business leaders, many told my the same investors to create demand sort of like return to office
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray Getting away from guilt will be hard given the training data and the lack of regard for copyright and licenses. Anyway, free software depends on copyright being honored for its licenses to work. Bubble or not, there are some vicious legal gotchas in there. And even small-scale use makes the climate crisis worse. It was already bad before this craze.
I'm not seeing a bright future for the stuff.
Happily, I love coding, so a lack of chatbot aid won't phase me. Not using it now, won't miss it tomorrow.
-
@bflipp @virtuous_sloth I'm curious: What kind of hardware is that?
I bought a used radeon 7900 off of ebay. I have a very old dual xeon supermicro motherboard with 256GB of DDR3 that it’s installed in. I can run much larger models than 30b but that’s the cutoff of for what I consider a close parity for my use cases between open source models (I use Qwen) and commercial ones like Opus, Sonnet, or GPT.
If you have a GPU with 16GB of VRAM and 32GB of system RAM you can have a decent experience for some use cases. I use it primarily as my Joplin interface for notes and task planning with OpenClaw, and for scripting/prototyping with Claude Code.
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray Let’s wait for this to happen. No need to have FOMO. If it comes, fine. If it doesn’t, finer.
-
I bought a used radeon 7900 off of ebay. I have a very old dual xeon supermicro motherboard with 256GB of DDR3 that it’s installed in. I can run much larger models than 30b but that’s the cutoff of for what I consider a close parity for my use cases between open source models (I use Qwen) and commercial ones like Opus, Sonnet, or GPT.
If you have a GPU with 16GB of VRAM and 32GB of system RAM you can have a decent experience for some use cases. I use it primarily as my Joplin interface for notes and task planning with OpenClaw, and for scripting/prototyping with Claude Code.
@bflipp @virtuous_sloth Linux?
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray the open source models are still built on theft and exploitation. They still reinforce social biases. Sure, much better than paying Anthropic.
-
@timbray weirdly I think contract law might also succeed for the same reason. Small market also.
@brandonscript @timbray Are people REALLY trusting it in contract law?
Software has the advantage that you can run a code-test-fix loop until it passes the tests. And then you can bug fix it later. It relies on understanding the code and having good tests, but it's theoretically feasible to "million monkeys, million typewriters" your code.
Contact law tends to need to be correct because otherwise the one time you need it (when someone challenges the contract or a dispute arises) then any mistakes are unknown but critical!
It feels like the crypto-bros have already shown the folly of "if we make contracts as software then it'll be perfect"… and then it's buggy and suddenly "no human arbitration, automated completion" is a flaw. And "GenAI writes contacts" seems to be only one step removed.
-
@timbray I do think there is a bubble that will pop, but I also think that useful AI is already escaping from the dev community via tooling like OpenClaw that is taking off and using lots of tokens automating tasks for people.
-
So those of you who are burning millions of tokens on Claude Code should probably be aware that you’re one of the few remaining forces inflating the bubble.
Whatever, it’ll pop anyhow, the #genAI space will become a tiny, mostly open-source, fraction of what it is today, and at that point. we devs can use it without feeling guilty.
@timbray Is there even enough ethically sourced content to produce clean open source AIs?
-
Yes I have not run a closed source OS on a desktop or laptop in a very long time.
This is a full Debian install but the end goal is eventually proxmox with ollama running containerized. Wanted to see it all work first before adding complexity and flexibility to the platform.
-
R relay@relay.infosec.exchange shared this topic
