So, my understanding of AI tokens -- it's a rate of measurement.
-
So, my understanding of AI tokens -- it's a rate of measurement. e.g.: my model on my hardware can perform at 5 tokens per second. Okay great. Now, WHEN THE FUCK did tokens become a monetary unit? Why is every tech-bro claiming that each token has value?
Let me compare this to another GPU based operation-- Hashcat / password cracking. When I use my rig to do @CrackMeIfYouCan contest at defcon, Lert's say I get 10Kh/s hash rate on Bcrypt2 (lol). Now, extrapolate this out; How do I pay an intern to do my work for me in hash rates?
Units of performance measurement is not money.
Lets dumb this down again. I have a car that can go 60MPH. I'll pay you in MILES PER HOUR.
???
they have played us for fools
-
So, my understanding of AI tokens -- it's a rate of measurement. e.g.: my model on my hardware can perform at 5 tokens per second. Okay great. Now, WHEN THE FUCK did tokens become a monetary unit? Why is every tech-bro claiming that each token has value?
Let me compare this to another GPU based operation-- Hashcat / password cracking. When I use my rig to do @CrackMeIfYouCan contest at defcon, Lert's say I get 10Kh/s hash rate on Bcrypt2 (lol). Now, extrapolate this out; How do I pay an intern to do my work for me in hash rates?
Units of performance measurement is not money.
Lets dumb this down again. I have a car that can go 60MPH. I'll pay you in MILES PER HOUR.
???
they have played us for fools
Look, I get the fact that it costs money to make a GPU go brrrrrrrrrrr.
And it takes more money to get more GPUs to do more tokens per second.
We should be measuring AI use in GPU hours, much like CPU time is calculated in top (yes, lol, it's totally accurate)
-
Look, I get the fact that it costs money to make a GPU go brrrrrrrrrrr.
And it takes more money to get more GPUs to do more tokens per second.
We should be measuring AI use in GPU hours, much like CPU time is calculated in top (yes, lol, it's totally accurate)
@kajer hey, do you wanna trade 1 CPU day on your current rig for 1.5 CPU days on my X230?
-
So, my understanding of AI tokens -- it's a rate of measurement. e.g.: my model on my hardware can perform at 5 tokens per second. Okay great. Now, WHEN THE FUCK did tokens become a monetary unit? Why is every tech-bro claiming that each token has value?
Let me compare this to another GPU based operation-- Hashcat / password cracking. When I use my rig to do @CrackMeIfYouCan contest at defcon, Lert's say I get 10Kh/s hash rate on Bcrypt2 (lol). Now, extrapolate this out; How do I pay an intern to do my work for me in hash rates?
Units of performance measurement is not money.
Lets dumb this down again. I have a car that can go 60MPH. I'll pay you in MILES PER HOUR.
???
they have played us for fools
They pay you in miles/ tokens/ hashes. This is an allotment, a future promise, the catch? They don't tell you what mph will be used when you travel/ the token processing rate/ the hash rate, or if these are arbitrarily set.
For all we know you could be traveling at 2 mph, slower than if you had walked. Even 1,000,000 tokens means nothing if they take 300 years to process.
No matter the way you slice or look at this, it makes zero sense, ultimately devaluing the worker.
-
@kajer hey, do you wanna trade 1 CPU day on your current rig for 1.5 CPU days on my X230?
God you're fucking awesome for this
-
They pay you in miles/ tokens/ hashes. This is an allotment, a future promise, the catch? They don't tell you what mph will be used when you travel/ the token processing rate/ the hash rate, or if these are arbitrarily set.
For all we know you could be traveling at 2 mph, slower than if you had walked. Even 1,000,000 tokens means nothing if they take 300 years to process.
No matter the way you slice or look at this, it makes zero sense, ultimately devaluing the worker.
Right! I have 0 control on how many tokens per second today's model is going to burn. Maybe "count to 10" is 3 tokens today, and 5 tomorrow?
As a homelabber, I can tell you exactly how many kw/h I burn when i run gpt-oss-20b. and how many kw/h oss-20b takes to "think" and lie to me. I also pay real $$ per kw/h that goes through my power meter. "Tokens" don't have monetary value.
I can "make" unlimited tokens on some model i downloaded from huggingface. Doesn't mean "I'm rich bitch!"
-
R relay@relay.infosec.exchange shared this topic
-
God you're fucking awesome for this
@rusty__shackleford @nyanbinary
The cpu powering my GPU rig is a celeronG, so yeah.
kajer@dogeminer02:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 60
model name : Intel(R) Celeron(R) CPU G1840 @ 2.80GHz
stepping : 3
microcode : 0x28
cpu MHz : 798.184
cache size : 2048 KB -
Right! I have 0 control on how many tokens per second today's model is going to burn. Maybe "count to 10" is 3 tokens today, and 5 tomorrow?
As a homelabber, I can tell you exactly how many kw/h I burn when i run gpt-oss-20b. and how many kw/h oss-20b takes to "think" and lie to me. I also pay real $$ per kw/h that goes through my power meter. "Tokens" don't have monetary value.
I can "make" unlimited tokens on some model i downloaded from huggingface. Doesn't mean "I'm rich bitch!"
@kajer @rusty__shackleford well, until you IPO with your "AI@home" initiative
But ok, I was actually rather confused what your problem is & now get it - tokens indeed aren't "joule" in the sense of being a invariant unit, I had kinda ignored that fact because I didn't even assume there would be any useful ability to compare between vendors/models/...
-
@rusty__shackleford @nyanbinary
The cpu powering my GPU rig is a celeronG, so yeah.
kajer@dogeminer02:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 60
model name : Intel(R) Celeron(R) CPU G1840 @ 2.80GHz
stepping : 3
microcode : 0x28
cpu MHz : 798.184
cache size : 2048 KB@kajer @rusty__shackleford ...fuck. Well, on the plus side, fans are broken on my thinkpad so... uh... you better manage that temperature carefully
-
@kajer @rusty__shackleford ...fuck. Well, on the plus side, fans are broken on my thinkpad so... uh... you better manage that temperature carefully
@nyanbinary @rusty__shackleford
LOL, that's a better analog for argument than I was reaching for!!!
