My stance on #LLM :
-
My stance on #LLM :
1. There _might_ be some useful use cases with this technology that could be worth exploring.
2. However, it is glaringly obvious that, as of now, their main purpose is to power the mother of all investments bubbles.
3. Which leads us to the present trillion dollar business case for "we must build energy- and water-wasting data centers everywhere so that we can scrape every single website a thousand times a month for new training data!"
4. Thus, there is currently pretty much no ethical way of using LLMs.
5. Any ethical exploration of LLM use cases will thus have to wait until the bubble has burst, the investors have moved on to the next scam, and we can sort through the rubble to check what is left.
this. Although I wonder if we’ll be able to train a new frontier model ever again once the bubble has burst (and frankly I don’t care if we can’t) -
@gatesvp how was your open model trained ?
@ced @juergen_hubert Sounds like you're suggesting that there is a specific model trainee regimen that you would consider to be ethical?
What does that look like?
-
@ced @juergen_hubert Sounds like you're suggesting that there is a specific model trainee regimen that you would consider to be ethical?
What does that look like?
One that's trained on specifically public domain/Creative Commons Zero source material.
As opposed to, say, the entirety of the World Wide Web.
-
@juergen_hubert I have to say that every use case I can imagine for them would be perfectly 100% ok on smaller local only models.
Actually, I would go so far as to say even most of the use-cases they're pretending they can do could be done local. They don't really need to be 300+GB models. That's just a way to not actually acquire properly clean data (eg just steal everything and cram it all together rather than paying people to actually create qualitative sets.)
So yeah, every single use-case basically blows away the current system anyway.
As you said, the current system exists only for supporting the bubble.
My hope is when this is all over communities make community collated data sets that are clean and lighter and never push them as being "general AI" which they are not.
@nazokiyoubinbou @juergen_hubert yeah it seems to me like they’re too preoccupied with chasing ever more marginal performance improvements instead of maybe considering how to make the models more efficient (bar that context compression thing recently). Evil tongues would claim it’s a competitive advantage for those with deeper pockets
-
@nazokiyoubinbou @juergen_hubert yeah it seems to me like they’re too preoccupied with chasing ever more marginal performance improvements instead of maybe considering how to make the models more efficient (bar that context compression thing recently). Evil tongues would claim it’s a competitive advantage for those with deeper pockets
@flying_saucers @nazokiyoubinbou
Also, the current approach places a disproportionate burden on those who maintain websites, as these get constantly scraped for new content and thus see drastically increased page loads.
Speaking as the maintainer of such a website.
-
@flying_saucers @nazokiyoubinbou
Also, the current approach places a disproportionate burden on those who maintain websites, as these get constantly scraped for new content and thus see drastically increased page loads.
Speaking as the maintainer of such a website.
@juergen_hubert @nazokiyoubinbou wouldn’t it be nice if people respected robots.txt
-
@juergen_hubert @nazokiyoubinbou wouldn’t it be nice if people respected robots.txt
@flying_saucers @nazokiyoubinbou
Instead, we get anonymous bot-nets with a changing roster of IP addresses.
Seriously. Within the span of six hours, my wiki once received 3,800 requests for "Special:RecentChanges". This is not something most readers will do.
-
@flying_saucers @nazokiyoubinbou
Instead, we get anonymous bot-nets with a changing roster of IP addresses.
Seriously. Within the span of six hours, my wiki once received 3,800 requests for "Special:RecentChanges". This is not something most readers will do.
@juergen_hubert that’s messed up. Is there a way to rate limit just that page in particular?
-
My stance on #LLM :
1. There _might_ be some useful use cases with this technology that could be worth exploring.
2. However, it is glaringly obvious that, as of now, their main purpose is to power the mother of all investments bubbles.
3. Which leads us to the present trillion dollar business case for "we must build energy- and water-wasting data centers everywhere so that we can scrape every single website a thousand times a month for new training data!"
4. Thus, there is currently pretty much no ethical way of using LLMs.
5. Any ethical exploration of LLM use cases will thus have to wait until the bubble has burst, the investors have moved on to the next scam, and we can sort through the rubble to check what is left.
@juergen_hubert "The investors have moved to the next scam".
Well, by this logic, there is no ethical existence in our current society.
-
@juergen_hubert @gatesvp @ced Which has been done! You have to put up with the model's capabilities being a couple years behind the state of the art, but then, if the state of the art can only be achieved by labor theft, maybe it shouldn't be the state of the art. https://www.engadget.com/ai/it-turns-out-you-can-train-ai-models-without-copyrighted-material-174016619.html
-
@juergen_hubert that’s messed up. Is there a way to rate limit just that page in particular?
Still trying to figure that out.
