The MRA test: If it looks like a MRA, swims like a MRA, and quacks like a MRA, then it probably is a MRA.

Gender only comes into it because men do this all the fucking time.

You'd be just as much of an ignorant moron correcting a man, and you'd fail to learn anything just the same. It's just got that special sprinkle of privilege on top when you do it to Dr Bender

Knock it off.

But in all these examples, "powerful" describes utility. A more powerful engine can do more things than a weaker one. A more advanced spreadsheet software can help users calculate/track more things than a less advanced one.

By that definition, wouldn't "powerful" for LLMs just mean less wrong, or wrong less often?

At the end of the day, a model is just predicting text. A model can't use a tool, but it can predict the text required to use a tool. A model can't write code, but it can predict the text that a compiler will turn into a program. We keep building integrations that allow tools to be driven by text, which allows text prediction models to 'use' them... but really it's still just predicting text.

The only real metrics that apply to an LLM are size, speed, and accuracy. For frontier models, size and speed is always compensated by throwing more hardware at it, so users never see it. Thus, the only reasonable measure of power, for the journalistic contexts you're talking about, is the accuracy of the text it's predicting.

Thus, a "more powerful AI model" is just one that is less wrong than the previous generations. No?

That said, I do agree with your points that journalists are doing the PR firms' jobs for them when they use "more powerful" as a stand-in for "less wrong."

These kinds of tools tend to amaze me. They are a step beyond. I think as they train on more tokens they are just seeing a side effect of things like this working better, being able to read documents and file tax forms. Agreed to be very skeptical because it’s marketing hype and Claude has clogged the internet with paid astroturfing. It’s disgusting.

Stitch - Design with AI

Stitch generates UIs for mobile and web applications, making design ideation fast and easy.

Stitch (stitch.withgoogle.com)

So I urge journalists and others who are tempted to describe models as "powerful" (or transcribe the PR copy of the companies that call them "powerful") to reflect on what you think that means, and what evidence you have that it is true.

/fin

They are advertised as having new and impressive functionalities (misleading called "capabilities") each time, which also seems like powerful in the spreadsheet/calculator sense ... or would be if they actually worked as advertised.

>>