<![CDATA[We knew, but the proof is nice.]]>We knew, but the proof is nice.

"Apple just proved that AI models cannot do math. Not advanced math. Grade school math. The kind a 10-year-old solves"

The guess-the-next-words machines don’t actually understand anything.

Verifying your browser | Nitter

favicon

(nitter.poast.org)

]]>https://board.circlewithadot.net/topic/6729f050-8780-48ad-a7a9-f189ba5bb6b8/we-knew-but-the-proof-is-nice.RSS for NodeThu, 09 Apr 2026 09:59:46 GMTTue, 07 Apr 2026 20:03:11 GMT60<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 20:33:03 GMT]]>@davidaugust When did they do this test? I tried it with the following LLMs: Sonnet 4.6, Codex 5.3, GPT-5.4, GPT-5-Mini and Kimi-K2.5. They all answer the kiwi question correctly.

]]>https://board.circlewithadot.net/post/https://mas.to/users/erwinrossen/statuses/116370989629985070https://board.circlewithadot.net/post/https://mas.to/users/erwinrossen/statuses/116370989629985070Wed, 08 Apr 2026 20:33:03 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 20:29:06 GMT]]>@davidaugust Of course an LLM cannot do math, but to be honest, that is also not what they're designed for. An LLM these days like Claude knows that it should take a calculator and type the equation in there, instead of hallucinating an answer. Complaining that an LLM can't do math is like complaining a screwdriver can't drill a hole.

You can counter that there are plenty of people who are using the screwdriver to drill the hole, but that is not on the tool, that is on the user.

]]>https://board.circlewithadot.net/post/https://mas.to/users/erwinrossen/statuses/116370974080787749https://board.circlewithadot.net/post/https://mas.to/users/erwinrossen/statuses/116370974080787749Wed, 08 Apr 2026 20:29:06 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 19:20:19 GMT]]>@drifthood @davidaugust This makes me think of "Clever Hans", the horse that appeared to do arithmetics but actually just responded to involuntary human cues:
https://en.wikipedia.org/wiki/Clever_Hans

]]>https://board.circlewithadot.net/post/https://androiddev.social/users/bladecoder/statuses/116370703633340298https://board.circlewithadot.net/post/https://androiddev.social/users/bladecoder/statuses/116370703633340298Wed, 08 Apr 2026 19:20:19 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 18:11:10 GMT]]>@davidaugust

October 2024

Link Preview Image
Apple Engineers Show How Flimsy AI ‘Reasoning’ Can Be

The new frontier in large language models is the ability to “reason” their way through problems. New research from Apple says it's not quite what it's cracked up to be.

favicon

WIRED (www.wired.com)

]]>https://board.circlewithadot.net/post/https://infosec.exchange/users/joriki/statuses/116370431723039208https://board.circlewithadot.net/post/https://infosec.exchange/users/joriki/statuses/116370431723039208Wed, 08 Apr 2026 18:11:10 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 17:52:59 GMT]]>@flq yes, many systems have tools and/or abilities built in to take over basic math operations that simpler LLMs failed at.

The salient and enduring issue, I think, is that the spin and marketing of LLMs as "understanding," "thinking" or "intelligent" (as those words typical meanings suggest) remains largely fictional.

]]>https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116370360217321541https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116370360217321541Wed, 08 Apr 2026 17:52:59 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 17:50:51 GMT]]>@pascal_le_merrer any day now. I hear potus say in two weeks.

]]>https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116370351813348209https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116370351813348209Wed, 08 Apr 2026 17:50:51 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 17:49:38 GMT]]>@audioflyer79 @alisynthesis @davidaugust how does it do if you swap the colors of the fruit?

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/morten_skaaning/statuses/116370347049501908https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/morten_skaaning/statuses/116370347049501908Wed, 08 Apr 2026 17:49:38 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 15:58:22 GMT]]>@davidaugust

Shortcut to paper: https://arxiv.org/pdf/2410.05229

]]>https://board.circlewithadot.net/post/https://fed.qaz.red/users/elithebearded/statuses/116369909531688068https://board.circlewithadot.net/post/https://fed.qaz.red/users/elithebearded/statuses/116369909531688068Wed, 08 Apr 2026 15:58:22 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 15:34:25 GMT]]>@davidaugust @scottjenson @xdydx

True. See @xdydx 's reply.

]]>https://board.circlewithadot.net/post/https://hachyderm.io/users/glitzersachen/statuses/116369815356257268https://board.circlewithadot.net/post/https://hachyderm.io/users/glitzersachen/statuses/116369815356257268Wed, 08 Apr 2026 15:34:25 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 14:56:04 GMT]]>@davidaugust interesting. Had to ask. Already fixed?

Link Preview Image
]]>https://board.circlewithadot.net/post/https://freiburg.social/users/flq/statuses/116369664569446078https://board.circlewithadot.net/post/https://freiburg.social/users/flq/statuses/116369664569446078Wed, 08 Apr 2026 14:56:04 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 14:29:00 GMT]]>@davidaugust AGI is coming son 🤭

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/pascal_le_merrer/statuses/116369558094072452https://board.circlewithadot.net/post/https://mastodon.social/users/pascal_le_merrer/statuses/116369558094072452Wed, 08 Apr 2026 14:29:00 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 13:48:58 GMT]]>@Karen5Lund Maybe because people stopped writing efficient code about 20 years ago?

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/bouriquet/statuses/116369400672671524https://board.circlewithadot.net/post/https://mastodon.social/users/bouriquet/statuses/116369400672671524Wed, 08 Apr 2026 13:48:58 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 13:17:17 GMT]]>@lemgandi
The wetness of water has been hotly debated, as to some wet means "covered with or soaked in water", and it's questioned whether water is covered with itself.
@davidaugust

]]>https://board.circlewithadot.net/post/https://mstdn.social/users/ozzelot/statuses/116369276105517171https://board.circlewithadot.net/post/https://mstdn.social/users/ozzelot/statuses/116369276105517171Wed, 08 Apr 2026 13:17:17 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 13:15:04 GMT]]>@davidaugust

Amo Bishop Rodent (@pikesley@mastodon.me.uk)

"We made the computers, the notoriously accurate calculating machines, worse at arithmetic. This is surely progress along the path to creating Computer God"

favicon

mastodon.me.uk (mastodon.me.uk)

]]>https://board.circlewithadot.net/post/https://mastodon.me.uk/users/pikesley/statuses/116369267373253824https://board.circlewithadot.net/post/https://mastodon.me.uk/users/pikesley/statuses/116369267373253824Wed, 08 Apr 2026 13:15:04 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 12:55:12 GMT]]>@alisynthesis @davidaugust fair enough. I changed up the problem completely and added some reasoning and it did pretty well. It appears to be generating code to solve the math. The only thing it missed is that very unripe bananas are green, not yellow.

James picks 40 apples on Monday. Then he picks 35 lemons on Tuesday. On Wednesday, he picks half as many bananas as he did apples, but five of them were very unripe. How many yellow fruits does James have?

Link Preview ImageLink Preview Image
]]>https://board.circlewithadot.net/post/https://mstdn.social/users/audioflyer79/statuses/116369189293853840https://board.circlewithadot.net/post/https://mstdn.social/users/audioflyer79/statuses/116369189293853840Wed, 08 Apr 2026 12:55:12 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 12:43:34 GMT]]>@audioflyer79 @davidaugust I mean, it's worth noting that the LLMs have ingested that paper by now. : /

]]>https://board.circlewithadot.net/post/https://io.waxandleather.com/users/alisynthesis/statuses/116369143527661164https://board.circlewithadot.net/post/https://io.waxandleather.com/users/alisynthesis/statuses/116369143527661164Wed, 08 Apr 2026 12:43:34 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 12:31:51 GMT]]>@davidaugust Ecosia AI gets it right. It looks like the paper referenced was published in 2025, so the research conducted prior. The models are all much better now. I’m no AI apologist, but I think any argument of “AI sucks because it’s not good at _____” is on tenuous ground and will be proven wrong as the models continue to improve. @Ecosia

Link Preview Image
]]>https://board.circlewithadot.net/post/https://mstdn.social/users/audioflyer79/statuses/116369097458043474https://board.circlewithadot.net/post/https://mstdn.social/users/audioflyer79/statuses/116369097458043474Wed, 08 Apr 2026 12:31:51 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 12:20:47 GMT]]>@joriki it’s from August.

]]>https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116369053962251543https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116369053962251543Wed, 08 Apr 2026 12:20:47 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 12:16:57 GMT]]>@drifthood yes, there does seem to be a threshold over which in some respects only humans cross over to one side.

I see that sort of begging in a dog. He wants the treat, so instead of just doing the desired behavior the human command is asking for, he tries every response that has ever gotten him a treat until he “unlocks” the treat. Humans can and do do this too from time to time, but humans _also_ actually communicate and understand from time to time as well.

]]>https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116369038868828654https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116369038868828654Wed, 08 Apr 2026 12:16:57 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 12:12:06 GMT]]>@Sobex it’s from August.

]]>https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116369019819477369https://board.circlewithadot.net/post/https://mastodon.online/users/davidaugust/statuses/116369019819477369Wed, 08 Apr 2026 12:12:06 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 09:22:07 GMT]]>@davidaugust In about 80 years we've gone from a room full of computers the size of refrigerators that were good at crunching numbers but not much else to computers the size of corporate office parks that can draw almost-convincing pictures of people with five fingers (and thumbs, too!) but can't do elementary school math.

And some people call this progress.

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/Karen5Lund/statuses/116368351371775536https://board.circlewithadot.net/post/https://mastodon.social/users/Karen5Lund/statuses/116368351371775536Wed, 08 Apr 2026 09:22:07 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 08:22:55 GMT]]>@davidaugust Direct link to the paper https://arxiv.org/pdf/2410.05229 (presented at ICLR 2025).

Seems not to be a very recent news, then.

]]>https://board.circlewithadot.net/post/https://social.sciences.re/users/Sobex/statuses/116368118601029127https://board.circlewithadot.net/post/https://social.sciences.re/users/Sobex/statuses/116368118601029127Wed, 08 Apr 2026 08:22:55 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 07:53:11 GMT]]>@davidaugust Well, there have actually been successes by connecting LLMs to proof assistant and computer algebra programs. As this post rightly puts, the LLM is not capable in itself to perform computations reliably, but it can write commands sent to the computer algebra programs, or proof candidates sent to the proof assistant; which can answer that the proof is incorrect, and the process goes on until a correct proof is produced.

See also uses by pro mathematicians:
https://bsky.app/profile/wildverzweigt.bsky.social/post/3miua4ulxhk2f

Also see Terence Tao

]]>https://board.circlewithadot.net/post/https://social.sciences.re/users/MonniauxD/statuses/116368001725262914https://board.circlewithadot.net/post/https://social.sciences.re/users/MonniauxD/statuses/116368001725262914Wed, 08 Apr 2026 07:53:11 GMT<![CDATA[Reply to We knew, but the proof is nice. on Wed, 08 Apr 2026 02:36:05 GMT]]>@davidaugust @glitzersachen

Actually, this particular joke has the attention of quite a few people..

Scott Jenson (@scottjenson@social.coop)

OK, this is going even MORE sideways so I need to make a few things clear: 1. I took a complex point and made it poorly 2. My goal was to ask for more inclusiveness 3. I am sickened by what happend to BlackTwitter and I don't want it recur 4. But I can't speak for BlackTwitter nor should I 5. I apologize to black mastodon users for making such a poor comparison 6. I'm not endorsing "AI Slop" they were a foil to make my point 7. I'm certainly NOT trying to compare AI bros to Black twitter (but, as I said, I can see how people made that connection. I'm trying to correct that here)

favicon

social.coop (social.coop)

@scottjenson

]]>https://board.circlewithadot.net/post/https://mastodon.social/users/xdydx/statuses/116366754793331832https://board.circlewithadot.net/post/https://mastodon.social/users/xdydx/statuses/116366754793331832Wed, 08 Apr 2026 02:36:05 GMT