The Algorithmic Bridge

The Algorithmic Bridge

Share this post

The Algorithmic Bridge
The Algorithmic Bridge
Science Says AI Isn't as Smart as You (Think)

Science Says AI Isn't as Smart as You (Think)

AI appears less smart when you look at this other chart

Alberto Romero's avatar
Alberto Romero
Jul 08, 2024
∙ Paid
21

Share this post

The Algorithmic Bridge
The Algorithmic Bridge
Science Says AI Isn't as Smart as You (Think)
7
5
Share

A blog about AI that’s actually about people

I will show you two graphs.

The first reflects AI’s improvement in categories like reading comprehension and speech recognition.1 It’s a well-known chart with a clear takeaway: Deep learning-based systems have overtaken humans in many cognitive areas.

Our World in Data

The second one is more recent. It depicts the growth trends of language models (i.e. chatbots) from the main three AI labs (OpenAI, Google DeepMind, and Anthropic) in average performance over the most-used benchmarks (MMLU, MATH, GPQA, TruthfulQA, GSM8k, etc.), which are harder than those depicted in the first graph.

Saw it from Jim Fan but can’t find the source

Although those two graphs emphasize different plotlines, they tell the same story: Over time, modern AI systems are getting better fast—so good that they’ve surpassed humans in skills we once thought exclusive to us.

This story is true; the measurements have been replicated many times.2

But like all stories, it is up to the storyteller’s discretion how to embellish it. In fiction, readers enjoy a little poetic license. In science, that’s a sin. It turns out the story those two graphs tell has become so adorned that the glittering attire conceals the plot.

There’s another version of this same story.

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Alberto Romero
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share