32 Comments
User's avatar
Babbage's avatar

Great post and I share a lot of the sentiments but there is something that weighs on the other side of the scales: that humans will understand a lot more about the universe than we ever thought possible.

Alberto Romero's avatar

I hope you are correct.

In any case, I don't think the current forces are *prioritizing* our understanding. It will be, at best, a secondary consideration. A byproduct of a world that progresses “despite us.” Why would humans doing the understanding be considered more important than humans doing the work?

Let's get over with this fleshy hindrance!

Venturing Somewhere's avatar

Despite what the most optimistic supporters may say, the priority is to win the AI race, make money, and ensure your success as a company. The natural goal is to make sure your operating at a level that wows first - whether it's experts, investors, or governments. That's inherently at odds with prioritizing our understanding

Gustavo José Zambrano's avatar

A compelling question that mirrors the architectural foundations I investigate in my essays: https://gzambrano.substack.com

Dom Stocchetti's avatar

Great piece. I suppose that with the frontier models we increasingly have a magic-making gun. What matters is where we choose to point it.

Gustavo José Zambrano's avatar

Great analytical depth. The relationship between entropy, information, and purposeful organization is endlessly fertile territory.

Sunil Malhotra's avatar

This is such a powerful post. There's no way though, that the vested will even understand, leave alone agree. The Big AI guys may be trying to read the label from inside the jar.

The irony is that they helped create the condition they now seem troubled by, and are perhaps trying to understand its consequences without stepping outside the system that produced them. “If you find yourself in a hole, stop digging” comes to mind.

The question may not be whether AI is destroying intellectual disciplines, but whether we are allowing it to change our relationship with knowing before we have understood what that relationship was worth. And that is precisely where the people building the systems may have the greatest difficulty seeing clearly: they are inside the transformation, not merely observing it.

A thought experiment — https://www.facebook.com/share/v/1Ewphp4Tya/

Alberto Romero's avatar

Thanks Sunil. I think they believe the benefits will outweigh the losses. That's why they keep going. It's a fair belief but, to me, not at all clear it's correct

Gustavo José Zambrano's avatar

An illuminating angle. When we dissect complex architectures, the implicit trade offs become remarkably visible.

ChatGPT for All's avatar

AI can generate math beyond human comprehension, but that doesn’t mean we understand it, it means discovery has outrun interpretation. logical bottleneck shifts from solving problems to building a connective layer that can verify, translate and actually use what the machine finds.

Gustavo José Zambrano's avatar

Exceptionally lucid writing. The real challenge is translating abstract theoretical principles into robust working habits.

Franck PARIENTI's avatar

Belle metaphore. On est fascine par le tour (les benchmarks, les demos) sans voir la mecanique derriere le rideau (les couts d'inference, les hallucinations, le travail humain de RLHF). Le vrai test d'un outil IA n'est pas la demo, c'est le deuxieme mois d'usage en production quand l'effet de surprise est passe.

JJ Emmanuel's avatar

I share some of your unease, but I think it's headed another direction. You fault Roon for false hope, then say you can see where this leads and there’s no way to stop it. I see you're also making a prophecy too then, just a darker one. It doesn't always go that way. I did some research for an essay on this stuff, I discovered in 1975 biologists at Asilomar paused their own genetic research until they’d agreed on safety rules. They didn’t stop it, but they offered some control over it. I am an old guy, it's just my thumbnail hasn't caught up. My strong intuition tells me that there is a backlash of powerful proportion against AI coming. Soon.

And I wonder whether “less human” assumes something we’ve never actually been. We were never the strongest or fastest creatures, and if machines can advance what thinking is, that doesn't make us less human. Maybe being human was never about being the best mind in the room. Maybe it was about witnessing and understanding what the show actually means.

Good read.

Abhishek Pan's avatar

A lot of those 300 are still being verified, and checking them moves at human speed. I'd expect mathematicians to spend the next year refereeing this pile before anyone knows how much of it holds.

Gustavo José Zambrano's avatar

This systemic paradox is at the heart of our ongoing research on second brains: https://gzambrano.substack.com

Mark's avatar

Alberto, much as I appreciate your writing, you are not qualified to have an opinion on this.

I look forward to your interview with Terence Tao. I'm sure that will be much more enlightening on what AI is contributing to the field of mathematics.

Alberto Romero's avatar

Feel free to read my other posts on the topic. All the references to authority you're looking for are there.

Gustavo José Zambrano's avatar

Fascinating reflection. True intellectual leverage comes from understanding what not to optimize prematurely.

Robert Ferrell's avatar

Why is the neutrino field left-chiral only? If an AI answered that, I might be able to understand.

Gustavo José Zambrano's avatar

A sharp and grounded critique. Reconciling algorithmic efficiency with human values remains the defining task of our era.

Charles's avatar

Are all your posts going to be depressing now? Were you an AI hype guy who never foresaw things turning out this way? Or did you think it would take longer?

Alberto Romero's avatar

"AI hype guy" you can go

Gustavo José Zambrano's avatar

Truly thoughtful perspective. The distinction between systemic complexity and intuitive heuristics here captures an essential reality of technological shifts.

E.A. Watling's avatar

Wasn't this result generated by mathematicians with the assistance of AI? Using models trained by great mathematicians to do great math? I guess I don't understand why that necessitates humans becoming a sideshow.

Alberto Romero's avatar

This is mostly just AI doing the work. It's used human data to train sure. The question is: what will happen when humans stop providing that data because there are no humans left doing math? I think that's an interesting question

Gustavo José Zambrano's avatar

Insightful reflection. The hardest problems in knowledge architecture always mirror cognitive constraints rather than purely computational limits.

Jeff Cook-Coyle's avatar

Alberto, the only response is to worship God. There is a Creator, a First Mover. He made everything, speaking it into existence. He has three parts, the Father, Son, and Holy Spirit. God made people, men and women, in His image. In His image means that He put us on earth in His place.

There were three primary (early) rebellions against God: in the Garden of Eden; precipitating the Great Flood; and by trying to build the Tower of Babel. Building the Tower of Babel had an important element of technological rebellion; building "a tower that reaches all the way to the heavens so that we can make a name for ourselves."

Quoting Dale Moreau's Substack, "The sin at Babel was not technological progress. It was theological revolt: humanity declaring that it could secure its own identity, its own permanence, and its own access to the divine without the God who gives all three."

It was a significant enough accomplishment that God destroyed the tower; dispersed the builders; inhibited them from communicating and collaborating; and appointed lesser gods over the new nations (the Gods of Nordic, Greek, Native American, and other cultures)

What is happening now is not news to Him. At what point does He need to respond? I don't know. What form will it take? I don't know that either. He may just let AI do something inherently self-limiting, like take down the electric grid.

But in all cases, community, worship, prayer, service, and Bible reading are the path through.

I say this not to evangelize but as a friend. Your insights are superb. You are gracious enough to share them. You work hard to express them well. I am appreciative, both of you and what you do. Now it's my turn to share.

Gustavo José Zambrano's avatar

A very perceptive observation. It is rare to see discussions connect foundational theory with practical reality so coherently.

User's avatar
Comment deleted
Oct 16
Comment deleted
Gustavo José Zambrano's avatar

A profound contribution. Mapping cognitive feedback loops requires exactly this kind of disciplined patience.