the idea that a "singularity" is possible is just the idea that you can turn "mistaking a sigmoid for an exponential" into a millenarian religion
-
seriously just imagine the plot of one of the movies that doomers seem to think are documentaries, like Terminator 2. imagine the scene where the T-1000 is getting pelted with bullets. instead of seamlessly autonomously healing, imagine it has to lie down and wait for a human to place an order for $1,000,000 of NVIDIA GPUs to be delivered in a shipping container and then a construction crew to set up a methane generator to run for two weeks straight before it got up again. is that still scary?
@glyph skynet was so intelligent, they built terminators so efficienly, they run on bare 6502s ; they dont even need nvidia GPUs.
LLMs are not even close.
-
doomers might look at my rant here and think, "but wait, once it's self-sustaining, even a little, it's TOO LATE, it's already out of control!!!" and to that I say: no. not even close. look the evolution of *any* business. managing resource flows is really hard. there is an off-ramp every single day
@glyph Another counterpoint: Every single zombie apocalypse scenario, where the collapse of human infrastructure and supply chains is so absolute, not even the zombies disappearing overnight would still lead to years, if not decades of recovery.
-
if, in order to achieve your out-of-control doomsday robot scenario, a trillion dollars worth of human effort must be expended annually, and if any of it stops for even a moment than the whole thing implodes and grinds to a halt, _you can stop worrying_ that it is "the machines" which dominate us
@glyph above all, if people believe singularity is scary, why the fuck do they invest a trillion $/yr to try to reach it ? At that point, we should try to convince them that a bigger CERN could really provoke a black hole on earth.
Won’t work either, but at least we’ll have something useful at the end for a fraction of the price! -
like if anyone had halfway-plausible "grey goo" nanotech that could do anything that looked like computation, that might be worrying. a locally viable self-reproducing platform that can make another one of itself from a pile of dirt, even if it's like, special dirt, that might scare me a little bit. but an overlord hive-mind that requires an uninterrupted global high-purity helium supply chain just to make ONE more of itself is supposed to be a threat?
@glyph Goddammit, this is twice in a row I'm forced to root for, of all things, the government of Iran.
Edit: For context, a lot of the world's helium trade goes through, you guessed it, the Strait of Hormuz.
-
the idea that a "singularity" is possible is just the idea that you can turn "mistaking a sigmoid for an exponential" into a millenarian religion
@glyph Believing LLM chatbots will achieve singularity is like someone believing teleportation and manufacture-anything-machines are right around the corner because they once saw a magician perform a magic trick.
-
@glyph my assertion was that the singularity, as described by ray kurzweil, accurately describes the invention of writing, and i don't see why it would be more interesting if the self-improving intelligent mechanism were made of etched silicon instead of CHNOPS nanomachines. it is harder for etched silicon to self-reproduce, anyway. the CHNOPS nanomachines just do that.
i think human advancement *has* followed an exponential-*looking* curve since that point, albeit with a low base.
@mcc @glyph
Language is a virus https://en.wikipedia.org/wiki/Language_is_a_Virus -
in order to be a singularity candidate, an AI would need to achieve vertical integration from silicon fabrication through logistics and integration, into operating systems and applications, with tight whole-system feedback from the robotics to the shipping to the power generation and back
@glyph This *strongly* depends on what you mean by "singularity". I think you're conflating that with "hard takeoff paperclips scenario" or something.
I can just barely (barely!) imagine a future where someone manages to use AI to get a more efficient form of AI, which would allow further bootstrapping without requiring more hardware. Same hardware gets more compute.
You're spot-on about the supply chain limitations, though. Good luck to the AI that wants to dig up more cobalt or whatever.
-
casual thinkpieces and lazy attempts at scicomm are what has set me off but the actual thing I'm mad about is that we are ruled by people with a child's understanding of the world and the economy and that's actually really bad
@glyph I’m not an AI doomer and I pretty much agree with your whole thread, but something to consider: when I think of AI taking over humanity, I see The Matrix more than Terminator. A hypothetical super intelligent AI wouldn’t necessarily need complete self-replication ability… it just needs the ability to influence humans enough to have them do the parts it can’t.
If you think about it, we’ve been building tech designed specifically to manipulate human behavior since the advent of social media… and it’s effective. If humanity were any good at protecting itself or organizing for the greater good, the perverse reward systems of capitalism would’ve been brought into check long before US oligarchs like Musk and Zuck could have been able to amass their power.
In a hypothetical world where machines rule, the more likely scenario is a majority of humans self-oppressing because they’ve been manipulated into it by adjustments to the algorithms that feed them the information they use to establish reality. We become part of the system.
The only real difference between that world and today is that today it’s a handful of billionaires controlling the algorithms. Replace Zuck and a few others with sufficiently capable AI and is it really that unbelievable that society would just keep cranking out more machines despite a slow degradation of quality of life?
Anyway - great thread and a fun topic to kick around. Thanks for posting it.
-
seriously just imagine the plot of one of the movies that doomers seem to think are documentaries, like Terminator 2. imagine the scene where the T-1000 is getting pelted with bullets. instead of seamlessly autonomously healing, imagine it has to lie down and wait for a human to place an order for $1,000,000 of NVIDIA GPUs to be delivered in a shipping container and then a construction crew to set up a methane generator to run for two weeks straight before it got up again. is that still scary?
@glyph This is all a component I hadn't really considered but entirely makes sense. And simultaneously reminds me of that scene in Ocean's Eleven where the one guy is trying to tell everybody why robbing a casino is a stupid fucking idea, capping the whole thing off with "and even if you *do* manage to pull all that off, *you're still in the middle of the fucking desert!*" (Which, tangentially, has become something of a metaphorical benchmark for me deciding whether it's worth doing some huge labor in general)
All that to say, awesome thread
-
@suetanvil it's ruining my ability to appreciate the fantasy!!!
That 'supposedly' is the clue.
(My guess: they are hardcore rationalists who took that position in order to flatter their own egos ("*I'm* too *smart* to believe in the Sky Bully") and so didn't think that position through. They are also so insulated from reality by their wealth that they don't know how to handle being wrong about something. So now they deal with the downsides of that choice by techbro-inventing religion from scrstch^W SF.)
-
That 'supposedly' is the clue.
(My guess: they are hardcore rationalists who took that position in order to flatter their own egos ("*I'm* too *smart* to believe in the Sky Bully") and so didn't think that position through. They are also so insulated from reality by their wealth that they don't know how to handle being wrong about something. So now they deal with the downsides of that choice by techbro-inventing religion from scrstch^W SF.)
(A techbro would rather sacrifice the world on the altar of his dark GPU gods than go to therapy.)
-
@glyph I've seen enough movies to know that the whole thing will come crashing down due to a very tiny inconsequential unnoticed design flaw. You know, like an expired SSL certificate.
-
-
-
@glyph This *strongly* depends on what you mean by "singularity". I think you're conflating that with "hard takeoff paperclips scenario" or something.
I can just barely (barely!) imagine a future where someone manages to use AI to get a more efficient form of AI, which would allow further bootstrapping without requiring more hardware. Same hardware gets more compute.
You're spot-on about the supply chain limitations, though. Good luck to the AI that wants to dig up more cobalt or whatever.
@varx imagining stuff is easy though. I can imagine lots of stuff
-
casual thinkpieces and lazy attempts at scicomm are what has set me off but the actual thing I'm mad about is that we are ruled by people with a child's understanding of the world and the economy and that's actually really bad
These children have the full backing of trillion dollar petrostate despots & oil oligarchs terrified of a fossil fuel phase out.
https://www.bloomberg.com/news/articles/2026-02-17/openai-anthropic-deals-power-abu-dhabi-s-100-billion-ai-betAI is the fossil fuel industry's ticket to keeping fossil fuel prices high.
https://www.businessinsider.com/microsoft-google-hand-dissident-data-to-saudi-arabia-activists-say-2023-7Gulf states want to be a global version of the Russian Internet Research Agency's hack-for-hire business model.
The AI singularity is just a euphemism for looting others assets.
1/
-
@glyph I’m not an AI doomer and I pretty much agree with your whole thread, but something to consider: when I think of AI taking over humanity, I see The Matrix more than Terminator. A hypothetical super intelligent AI wouldn’t necessarily need complete self-replication ability… it just needs the ability to influence humans enough to have them do the parts it can’t.
If you think about it, we’ve been building tech designed specifically to manipulate human behavior since the advent of social media… and it’s effective. If humanity were any good at protecting itself or organizing for the greater good, the perverse reward systems of capitalism would’ve been brought into check long before US oligarchs like Musk and Zuck could have been able to amass their power.
In a hypothetical world where machines rule, the more likely scenario is a majority of humans self-oppressing because they’ve been manipulated into it by adjustments to the algorithms that feed them the information they use to establish reality. We become part of the system.
The only real difference between that world and today is that today it’s a handful of billionaires controlling the algorithms. Replace Zuck and a few others with sufficiently capable AI and is it really that unbelievable that society would just keep cranking out more machines despite a slow degradation of quality of life?
Anyway - great thread and a fun topic to kick around. Thanks for posting it.
@mathaetaes @glyph This. The scary part is the brainrotted human fans rushing to tear down millennia of progress in a decade, not anything technical.
-
These children have the full backing of trillion dollar petrostate despots & oil oligarchs terrified of a fossil fuel phase out.
https://www.bloomberg.com/news/articles/2026-02-17/openai-anthropic-deals-power-abu-dhabi-s-100-billion-ai-betAI is the fossil fuel industry's ticket to keeping fossil fuel prices high.
https://www.businessinsider.com/microsoft-google-hand-dissident-data-to-saudi-arabia-activists-say-2023-7Gulf states want to be a global version of the Russian Internet Research Agency's hack-for-hire business model.
The AI singularity is just a euphemism for looting others assets.
1/
2/
Oil profits fund the AI spyware & election meddling necessary for ubiquitous surveillance.
As the planet fries, oil oligarchs need a means to quell dissent & revolutions while laundering their money.
https://popular.info/p/billionaire-surveillance-enthusiastBillionaire Larry Ellison says a vast AI-fueled surveillance system can ensure 'citizens will be on their best behavior'
Billionaire Oracle cofounder Larry Ellison said he expects AI surveillance systems to reach a point where all citizens are under constant watch.
Business Insider (www.businessinsider.com)
The Terrifying Tentacles of Paramount’s Media Empire
With Paramount’s Warner Bros. Discovery takeover, the Ellisons are building a multicorporate empire of data mining, surveillance, news, and entertainment. What could possibly go wrong?
(www.theringer.com)
They're laundering their cash in a rush, via American tech, before the citizens of those countries realize how they're being looted & pillaged and launch revolutions like the Arab Spring.
Heavy lies the crown: The survival of Arab monarchies, 10 years after the Arab Spring | Brookings
Arab monarchs may find that their usual containment strategies will prove less effective as populations and opposition movements transform, and that the time has come for them to adapt their behavior.
Brookings (www.brookings.edu)
-
2/
Oil profits fund the AI spyware & election meddling necessary for ubiquitous surveillance.
As the planet fries, oil oligarchs need a means to quell dissent & revolutions while laundering their money.
https://popular.info/p/billionaire-surveillance-enthusiastBillionaire Larry Ellison says a vast AI-fueled surveillance system can ensure 'citizens will be on their best behavior'
Billionaire Oracle cofounder Larry Ellison said he expects AI surveillance systems to reach a point where all citizens are under constant watch.
Business Insider (www.businessinsider.com)
The Terrifying Tentacles of Paramount’s Media Empire
With Paramount’s Warner Bros. Discovery takeover, the Ellisons are building a multicorporate empire of data mining, surveillance, news, and entertainment. What could possibly go wrong?
(www.theringer.com)
They're laundering their cash in a rush, via American tech, before the citizens of those countries realize how they're being looted & pillaged and launch revolutions like the Arab Spring.
Heavy lies the crown: The survival of Arab monarchies, 10 years after the Arab Spring | Brookings
Arab monarchs may find that their usual containment strategies will prove less effective as populations and opposition movements transform, and that the time has come for them to adapt their behavior.
Brookings (www.brookings.edu)
3/
How The Arab Spring Affects Saudi Society
Revolt continues to pressure governments along Saudi Arabia's borders. Saudi Arabia experts Bernard Haykel and Toby Jones explain the effect of the Arab Spring on Saudi society and government, and what the turmoil across the Middle East and North Africa may mean for the U.S.-Saudi relationship.
NPR (www.npr.org)
https://www.nytimes.com/2026/01/08/world/europe/trump-lauder-ukraine-lithium.html
That's why Trump attacked Iran. Before the uprising by the young people of Iran could get ahead & the covert sale of sanctioned oil was revealed.
https://popular.info/p/update-after-sending-billions-toTrump's fossil fuel overlords don't want examinations of the fossil corruption looting every nation's national treasuries.
https://popular.info/p/kushner-breaks-pledge-seeks-5-billionGulf royals are looting like Putin's oligarchs loot Ukraine & Hungary and AI launders the cash.
Venture capital firm hit with $216 million penalty for ‘egregious’ violations of US sanctions against Russian billionaire - ICIJ
The Office of Foreign Assets Control said the firm managed a $20 million investment for Suleiman Kerimov, a member of Putin’s inner circle, after he was sanctioned in 2018.
International Consortium of Investigative Journalists (www.icij.org)
-
3/
How The Arab Spring Affects Saudi Society
Revolt continues to pressure governments along Saudi Arabia's borders. Saudi Arabia experts Bernard Haykel and Toby Jones explain the effect of the Arab Spring on Saudi society and government, and what the turmoil across the Middle East and North Africa may mean for the U.S.-Saudi relationship.
NPR (www.npr.org)
https://www.nytimes.com/2026/01/08/world/europe/trump-lauder-ukraine-lithium.html
That's why Trump attacked Iran. Before the uprising by the young people of Iran could get ahead & the covert sale of sanctioned oil was revealed.
https://popular.info/p/update-after-sending-billions-toTrump's fossil fuel overlords don't want examinations of the fossil corruption looting every nation's national treasuries.
https://popular.info/p/kushner-breaks-pledge-seeks-5-billionGulf royals are looting like Putin's oligarchs loot Ukraine & Hungary and AI launders the cash.
Venture capital firm hit with $216 million penalty for ‘egregious’ violations of US sanctions against Russian billionaire - ICIJ
The Office of Foreign Assets Control said the firm managed a $20 million investment for Suleiman Kerimov, a member of Putin’s inner circle, after he was sanctioned in 2018.
International Consortium of Investigative Journalists (www.icij.org)
4/
Russia offered U.S. a deal for minerals in Ukrainian territory it seized
Russian officials proposed to Trump administration officials an agreement for the U.S. to make money off critical minerals and metals under Moscow’s control.
NBC News (www.nbcnews.com)
The financial services industry of the UK & USA are always willing to lend fascists a hand.
https://nationalpost.com/news/how-six-months-before-the-second-world-war-britain-gave-hitler-9-million-in-gold-that-belonged-to-another-country
Documents reveal Bank of England sold stolen gold for Nazis
Archived material details how gold bars plundered from Czechoslovakia were sold on behalf of Germany's central bank in 1939
The Independent (www.independent.co.uk)
How Bank of England 'helped Nazis sell gold stolen from Czechs'
Official account of what many believe was British central bank's most shameful episode revealed more than 70 years after event
the Guardian (www.theguardian.com)
Russian invasions provide a handy excuse for delay of restitution
https://www.theguardian.com/world/2018/aug/19/russia-warsaw-pact-1968-invasion-czechoslovakia
Nazi Gold: Why Hitler's Germany was Desperate for Bullion
During World War II, the German Nazi government robbed more than 600 tons of European gold
TIME (time.com)
