The Algorithmic Bridge

The Algorithmic Bridge

Share this post

The Algorithmic Bridge
The Algorithmic Bridge
ChatGPT Will *Always* Make Stuff Up—What Can We Do About It?
Copy link
Facebook
Email
Notes
More

ChatGPT Will *Always* Make Stuff Up—What Can We Do About It?

Demystifying generative AI #4

Alberto Romero's avatar
Alberto Romero
Jun 14, 2023
∙ Paid
13

Share this post

The Algorithmic Bridge
The Algorithmic Bridge
ChatGPT Will *Always* Make Stuff Up—What Can We Do About It?
Copy link
Facebook
Email
Notes
More
6
3
Share
Midjourney

The hallucination problem (also called confabulation, my preferred term, fabrication, or simply making up stuff) refers to the tendency of language models (LMs) to generate text that deviates from what’s objectively true (e.g., ChatGPT saying 2 + 2 = 5 or GPT-3 implying Steve Jobs was still alive in 2020).

Although confabulation is pervasive—and a no-go when factuality is required—it doesn't matter in some cases. ChatGPT is great for tasks where truthfulness isn't relevant (e.g., idea generation) or mistakes can be assessed and corrected (e.g., reformatting). When boundless creativity is central (e.g., world-building) confabulation is even welcome.

High-stakes categories like finance and healthcare, however, require precision and reliability. Thankfully, the latest research suggests we're on the right track: GPT-4 was a jump from GPT-3, which already improved over GPT-2… the trend is clear; AI gets more factual over time. Extending this assumption we get that LMs will eventually reach a point where they confabulate less than humans—and not much later they won't confabulate at all.

But maybe not. So far, success has been partial. Instruction tuning, RLHF, Constitutional AI (RLAIF), prompt techniques like Chain-of-Thought and Tree of Thoughts, etc. alleviate but don’t eliminate confabulation. It's neither only a matter of data, compute, or size. Confabulation is the symptom of a design choice only fully modifiable with a design overhaul.

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Alberto Romero
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More