Right? It bothers me that the simplicity of such expressions appeals so much to people (thank you, Dennett, for the deepity concept that just as easily explains why it's wrong)
Yes! I've interpreted that attitude toward AI as a way for people to put off the thought of the potential implications of it being more than mere "math and engineering." A temporary cognitive fix.
Thanks Alberto for this insightful view! Our brain as well as artificial networks are capable of nonlinear interactions of distinct units, and this indeed can cause emergent behavior.
People want to squeeze a complex, multidimenisonal, system down to a simple line with us, at one end, and a rock, at the other. Once you've made that basic mistake it becomes easy to argue for whichever of these limits you feel most emotionally attached to.
(I've recently read some stuff arguing that people view other people the same way - Someone is either good or bad, but never good in some respects and bad in others)
100% and that's the exact reason why people do this. Those kinds of reductionist definitions are perfect to believe what they already want to believe (it indeed applies to how we see people, especially those we don't know personally and have no bond with)
What you're asking is whether I think a machine can be conscious? Or if just humans (carbon-based intelligence) can? I really don't know haha and have no criterion to choose other than what feels intuitive (bad criterion) or what I want it to be (also bad). But, if I had to guess, I don't see why consciousness would be exclusive to us.
Wow Alberto, that's one of the best essays I've read on what AI could be: something we cannot understand or make sense of. With our simplistic analogies to 'consciousness,' algebra, and the human brain, we are looking in the wrong directions. It's the same myopia that leads us to look for water on distant planets because it could mean 'life,' as if other forms of existence could only be organic. We are at the same time SO ignorant and so full of ourselves.
This is all well and good, but can you make the same case without setting up a straw man? Because while AI might not be "just" linear algebra, it most certainly is linear algebra. Given a pretrained model, the outputs are derived from the inputs via nothing more than a series of matrix add/multiply operations. If there is something "more" to this (and I am not saying that there isn't) it should be possible to specify what that is in technical terms. I'm not convinced that Anna Karenina (or any other creative work) sheds any light on this. At this point the jury is still out - the "it's just linear algebra" still have an arguable case.
You're the reason why I had to write this post. Thank you. (Now, please, specify in technical terms how consciousness emerges from brain activity or how intelligence emerges from chemistry.)
What has consciousness got to do with it? And I am not persuaded that there is a valid analogy to be made between AI/math and human intelligence/chemistry.
Consider a pretrained generative large language model. It's internal state, i.e. the mathematical operations that it will perform on any given input, are fixed. From the given input, it generates a corresponding deterministic output comprising a vector of next-token probabilities for every token in its vocabulary. One of these tokens is chosen, added to the input sequence, and then the process is repeated until some stopping condition is satisfied.
If the most probable token is chosen every time (equivalent, in most configurations, to setting the 'temperature' parameter to zero) then the entire output from a given input is entirely deterministic, derived from the frozen model state and the input. For example, running Mistral-Nemo 12B locally under Ollama, if I use the system prompt "You are a novelist. Take the text provided by the user and continue the story.", set temperature to zero, and provide the user input "It is a truth universally acknowledged...", the model reliably and repeatably reproduces the remainder of the first sentence of Pride and Prejudice, followed by a rather dull summary of the novel's romantic narrative of Mr Darcy and Elizabeth Bennet.
As I turn up the temperature, the output ceases to be repeatable, because each further token is being chosen increasingly at random. But even so, the model generates variations on the characters and plot of Pride and Prejudice. Eventually (at around a temperature of 1.1 in my experiments, in which I have also effectively maxed out top_k and top_p so that all tokens are available in principle) the model transitions rapidly from variations on the plot of the novel in perfect English to complete gibberish in a mixture of languages!
This is a rather disappointing form of 'intelligence'. Without added (pseudo)randomness it is not creative at all. With a little randomness it becomes more creative, but still can't go anywhere with 'It is a truth universally acknowledged...' other than Pride and Prejudice. Why not? There are surely lots of universally acknowledged truths. It is a truth universally acknowledged that a zombie in possession of brains must be in want of more brains. (Yes, that's the opening sentence of the parody novel Pride and Prejudice and Zombies, written by a human.) And with additional randomness it never starts to diverge creatively from Pride and Prejudice; it just suddenly loses the plot completely.
A reductionist might argue here that perhaps human intelligence and creativity is just the result of natural randomness. After all, chemical reactions are affected by the environment in which they occur, which is inevitably uncontrolled and noisy. And even if it weren't, perhaps we could fall back on quantum uncertainty as our source of randomness. But even if that's true, it's hardly the same thing as our generative AI model. The randomness in nature is everywhere, not just in a feedback loop that selects the next 'idea token'. There's no useful or meaningful analogy to be drawn.
My take is that our current generative models are interesting and they are useful. I use them all the time. By scaling them up, and wrapping them in clever applications and prompting techniques we can make them more interesting and useful... up to a point at least (which we seem not yet to have reached). But I think they're a dead end as far as a path to AGI is concerned.
'It's just linear algebra' might be a slight oversimplification. Nevertheless, artificial neural networks (ANNs) are well defined mathematical functions. And concepts like emergence and intelligence simply lose all meaning when we apply them to abstract/mathematical/platonic objects. That is why people can not meaningfully talk about ANNs as intelligent systems without retorting to anthropomorphic concepts or analogies with biological neurons, which fall apart instantly under close inspection. The idea that ANNs consist of 'digital neurons' interacting with each other being one of them.
We have to anthropomorphize because we don't have yet a good conceptualization for these things. I agree that's not the best approach. Just don't compare them to human minds. The last five sentences respond to your concerns!
I think on the contrary we have an exact unambiguous mathematical description of ANNs. In any other natural science such a complete description would be the end of the story. The problem only arises when people try to impose conceptions of how the human mind or biological neural networks work onto ANNs. I have a few posts on subject that discuss these in more detail (e.g. https://aichats.substack.com/p/the-black-box-part-i?r=4tn68o).
Yes, I have (physicist by training). However, in AI Anderson is wildly misrepresented to the point where one would think that he said 'More is magic'. The paper in it's essence is a discussion of symmetry breaking and emergence in the physicalist sense. In fact Anderson explicitly says 'As I said, we must all start with reductionism, which I fully accept' in the paper. That being said the concept of emergence as it is understood in physics/natural sciences simply does not apply to modern feedforward ANNs since ANNs are static functions with no internal dynamics or interactions.
Ok, so you read it and understand it but are emotionally repulsed by AI being more than just math. I prefer to be open-minded. And I think Anderson would be as well.
I don't think it is controversial that ANNs, which covers almost all of current AI, are mathematical functions. I would be absurd to be repulsed by AI - the irrationality of the current AI discourse is another matter.
I never said your argument was wrong. I agree computers are essentially magical rocks and AI exists in that realm. That said i think it’s important to note that AI is not a being, it is a doing. I am skeptical of who does the doing and why.
Will we get a jet engine by scaling a steam engine? Was the brain of Albert Einstein three times as large as the brain of an ordinary human? Yes, AI is more than matrix multiplication and probably more that the infamous stocastic parrot. But is it intelligence? Is it thinking? Is it even capable of reacting to our of distribution events? Will it interpolate when confronted with new I formation like a machine or Will it extrapolate like an sentient entity? Will we get AGI over the next 24 months by scaling transformers and giving them more inference time and call that thinking? I'm wondering if current frontier AI ndeeds to scale complexity and start to develop new paradigms to reach anything meaningful breakthrough. If GPT-4.5 would have shown the same jump in capacity as from 3.5 to 4 we wouldn't even have the discussion. I'm certain that current AI has more in common with the working of an virus than a brain:
I wrote elsewhere about the misunderstandings with GPT-4.5. The rest, I agree. I never went into specifics. I said what I said, which you seem to agree with!
I understand the idea you are trying to convey Alberto, and it is conveyed well.
However, like other previous commenters, I think the ‘matrix multiplication‘ is oversimplified. Similar to saying ‘computers work with zeros and ones’, true but not useful. What I would have used is ‘AI is mostly statistical analysis’ which is true and also meaningful. Using statistical analysis to try to create logic is the wrong direction for software development in my opinion (that’s why LLMs can’t count r’s in strawberry). Would the article still make sense if it used this analogy instead? Probably not.
It would. But regardless, people go as far as to reduce AI to "just linear algebra" because the goal is not to be faithful but to create a memetics effect (linesr algebra is easier to conceptualize than statistical analysis, which is too ambiguous). (I understand you have a symbolic background or at least that you support a neuro-symbolic approach to AI. I don't oppose that, I'm open to that possibility.)
I see the through line in this piece about the absurdity of comparing AI to “just linear algebra” and your earlier comments about critics being profoundly polarized into the hype and anti hype camps… and ultimately wrong through lazy thinking: people need cognitive shortcuts to dismiss what we have no time to care about because we’re so overloaded with information.
To me, it seems significant these realizations are becoming more widespread as AI gets more widely used. It’s a tool that does a great job connecting disparate information—it basically operates exclusively in the realm of metaphor, of similar “distances” between the vibes of different tokens. It also kills summarizing complex articles or my stream of consciousness ramblings into nice ordered lists with pretty headers. Making order out of chaos.
Someone somewhere who is smarter than me is hopefully asking how to harness the power of AI toward moving us into more liminality, nuance, and complexity, given that instead of 1’s and 0’s it operates on a matrix of relationships between different points in an n dimensional space. It exists in the squishy in-between. It operates more like the universe does, through order arising via the interplay of relationships.
It ought to be better at connecting us than the binary choice between Like/Dislike that was offered by social media, the previous media revolution.
I think that's just human nature. AI won't change that haha. It's not social media or information overload rather those are consequences of how we already were. Mercier and Sperber found this to be true: we're not scientists at heart but lawyers and politicians. We don't seek truth but predictability. We want the universe to be intelligible and not at all surprising. We believe what we want to believe because revisiting preconceptions requires a lot of mental effort. And the brain, for all its virtues, it's very good at tricking us.
Thank you for putting into words why “AI is just linear algebra,” or “It’s just a sophisticated token prediction machine,” always felt so wrong
Right? It bothers me that the simplicity of such expressions appeals so much to people (thank you, Dennett, for the deepity concept that just as easily explains why it's wrong)
Yes! I've interpreted that attitude toward AI as a way for people to put off the thought of the potential implications of it being more than mere "math and engineering." A temporary cognitive fix.
Thanks Alberto for this insightful view! Our brain as well as artificial networks are capable of nonlinear interactions of distinct units, and this indeed can cause emergent behavior.
Short and sweet way to refute anyone dismissive of AI 👍
People want to squeeze a complex, multidimenisonal, system down to a simple line with us, at one end, and a rock, at the other. Once you've made that basic mistake it becomes easy to argue for whichever of these limits you feel most emotionally attached to.
(I've recently read some stuff arguing that people view other people the same way - Someone is either good or bad, but never good in some respects and bad in others)
100% and that's the exact reason why people do this. Those kinds of reductionist definitions are perfect to believe what they already want to believe (it indeed applies to how we see people, especially those we don't know personally and have no bond with)
Thank you, I enjoyed reading that. When it comes to intelligence would you say you are a compatibilist or a functionalist?
What you're asking is whether I think a machine can be conscious? Or if just humans (carbon-based intelligence) can? I really don't know haha and have no criterion to choose other than what feels intuitive (bad criterion) or what I want it to be (also bad). But, if I had to guess, I don't see why consciousness would be exclusive to us.
Inspiring stuff. People really like to be right about what they already “know.”
The illusion of certainty is worse than ignorance
Wow Alberto, that's one of the best essays I've read on what AI could be: something we cannot understand or make sense of. With our simplistic analogies to 'consciousness,' algebra, and the human brain, we are looking in the wrong directions. It's the same myopia that leads us to look for water on distant planets because it could mean 'life,' as if other forms of existence could only be organic. We are at the same time SO ignorant and so full of ourselves.
We really know nothing
This is all well and good, but can you make the same case without setting up a straw man? Because while AI might not be "just" linear algebra, it most certainly is linear algebra. Given a pretrained model, the outputs are derived from the inputs via nothing more than a series of matrix add/multiply operations. If there is something "more" to this (and I am not saying that there isn't) it should be possible to specify what that is in technical terms. I'm not convinced that Anna Karenina (or any other creative work) sheds any light on this. At this point the jury is still out - the "it's just linear algebra" still have an arguable case.
You're the reason why I had to write this post. Thank you. (Now, please, specify in technical terms how consciousness emerges from brain activity or how intelligence emerges from chemistry.)
What has consciousness got to do with it? And I am not persuaded that there is a valid analogy to be made between AI/math and human intelligence/chemistry.
Consider a pretrained generative large language model. It's internal state, i.e. the mathematical operations that it will perform on any given input, are fixed. From the given input, it generates a corresponding deterministic output comprising a vector of next-token probabilities for every token in its vocabulary. One of these tokens is chosen, added to the input sequence, and then the process is repeated until some stopping condition is satisfied.
If the most probable token is chosen every time (equivalent, in most configurations, to setting the 'temperature' parameter to zero) then the entire output from a given input is entirely deterministic, derived from the frozen model state and the input. For example, running Mistral-Nemo 12B locally under Ollama, if I use the system prompt "You are a novelist. Take the text provided by the user and continue the story.", set temperature to zero, and provide the user input "It is a truth universally acknowledged...", the model reliably and repeatably reproduces the remainder of the first sentence of Pride and Prejudice, followed by a rather dull summary of the novel's romantic narrative of Mr Darcy and Elizabeth Bennet.
As I turn up the temperature, the output ceases to be repeatable, because each further token is being chosen increasingly at random. But even so, the model generates variations on the characters and plot of Pride and Prejudice. Eventually (at around a temperature of 1.1 in my experiments, in which I have also effectively maxed out top_k and top_p so that all tokens are available in principle) the model transitions rapidly from variations on the plot of the novel in perfect English to complete gibberish in a mixture of languages!
This is a rather disappointing form of 'intelligence'. Without added (pseudo)randomness it is not creative at all. With a little randomness it becomes more creative, but still can't go anywhere with 'It is a truth universally acknowledged...' other than Pride and Prejudice. Why not? There are surely lots of universally acknowledged truths. It is a truth universally acknowledged that a zombie in possession of brains must be in want of more brains. (Yes, that's the opening sentence of the parody novel Pride and Prejudice and Zombies, written by a human.) And with additional randomness it never starts to diverge creatively from Pride and Prejudice; it just suddenly loses the plot completely.
A reductionist might argue here that perhaps human intelligence and creativity is just the result of natural randomness. After all, chemical reactions are affected by the environment in which they occur, which is inevitably uncontrolled and noisy. And even if it weren't, perhaps we could fall back on quantum uncertainty as our source of randomness. But even if that's true, it's hardly the same thing as our generative AI model. The randomness in nature is everywhere, not just in a feedback loop that selects the next 'idea token'. There's no useful or meaningful analogy to be drawn.
My take is that our current generative models are interesting and they are useful. I use them all the time. By scaling them up, and wrapping them in clever applications and prompting techniques we can make them more interesting and useful... up to a point at least (which we seem not yet to have reached). But I think they're a dead end as far as a path to AGI is concerned.
It is just math & we can do really cool stuff with math. I’m a proud reductionist and won’t be silenced by the merchants of complexity!
'It's just linear algebra' might be a slight oversimplification. Nevertheless, artificial neural networks (ANNs) are well defined mathematical functions. And concepts like emergence and intelligence simply lose all meaning when we apply them to abstract/mathematical/platonic objects. That is why people can not meaningfully talk about ANNs as intelligent systems without retorting to anthropomorphic concepts or analogies with biological neurons, which fall apart instantly under close inspection. The idea that ANNs consist of 'digital neurons' interacting with each other being one of them.
We have to anthropomorphize because we don't have yet a good conceptualization for these things. I agree that's not the best approach. Just don't compare them to human minds. The last five sentences respond to your concerns!
I think on the contrary we have an exact unambiguous mathematical description of ANNs. In any other natural science such a complete description would be the end of the story. The problem only arises when people try to impose conceptions of how the human mind or biological neural networks work onto ANNs. I have a few posts on subject that discuss these in more detail (e.g. https://aichats.substack.com/p/the-black-box-part-i?r=4tn68o).
Have you read the "more is different" paper?
Yes, I have (physicist by training). However, in AI Anderson is wildly misrepresented to the point where one would think that he said 'More is magic'. The paper in it's essence is a discussion of symmetry breaking and emergence in the physicalist sense. In fact Anderson explicitly says 'As I said, we must all start with reductionism, which I fully accept' in the paper. That being said the concept of emergence as it is understood in physics/natural sciences simply does not apply to modern feedforward ANNs since ANNs are static functions with no internal dynamics or interactions.
Ok, so you read it and understand it but are emotionally repulsed by AI being more than just math. I prefer to be open-minded. And I think Anderson would be as well.
I don't think it is controversial that ANNs, which covers almost all of current AI, are mathematical functions. I would be absurd to be repulsed by AI - the irrationality of the current AI discourse is another matter.
It’s not just linear algebra. It’s also a scam to make a bunch of white guys rich by devaluing the working class.
Incredible. Can you engage with the arguments without losing control of your emotions?
I never said your argument was wrong. I agree computers are essentially magical rocks and AI exists in that realm. That said i think it’s important to note that AI is not a being, it is a doing. I am skeptical of who does the doing and why.
Then we agree
Will we get a jet engine by scaling a steam engine? Was the brain of Albert Einstein three times as large as the brain of an ordinary human? Yes, AI is more than matrix multiplication and probably more that the infamous stocastic parrot. But is it intelligence? Is it thinking? Is it even capable of reacting to our of distribution events? Will it interpolate when confronted with new I formation like a machine or Will it extrapolate like an sentient entity? Will we get AGI over the next 24 months by scaling transformers and giving them more inference time and call that thinking? I'm wondering if current frontier AI ndeeds to scale complexity and start to develop new paradigms to reach anything meaningful breakthrough. If GPT-4.5 would have shown the same jump in capacity as from 3.5 to 4 we wouldn't even have the discussion. I'm certain that current AI has more in common with the working of an virus than a brain:
https://open.substack.com/pub/theafh/p/what-viruses-can-teach-us-about-ai?utm_source=share&utm_medium=android&r=42gt5
I wrote elsewhere about the misunderstandings with GPT-4.5. The rest, I agree. I never went into specifics. I said what I said, which you seem to agree with!
I understand the idea you are trying to convey Alberto, and it is conveyed well.
However, like other previous commenters, I think the ‘matrix multiplication‘ is oversimplified. Similar to saying ‘computers work with zeros and ones’, true but not useful. What I would have used is ‘AI is mostly statistical analysis’ which is true and also meaningful. Using statistical analysis to try to create logic is the wrong direction for software development in my opinion (that’s why LLMs can’t count r’s in strawberry). Would the article still make sense if it used this analogy instead? Probably not.
It would. But regardless, people go as far as to reduce AI to "just linear algebra" because the goal is not to be faithful but to create a memetics effect (linesr algebra is easier to conceptualize than statistical analysis, which is too ambiguous). (I understand you have a symbolic background or at least that you support a neuro-symbolic approach to AI. I don't oppose that, I'm open to that possibility.)
I see the through line in this piece about the absurdity of comparing AI to “just linear algebra” and your earlier comments about critics being profoundly polarized into the hype and anti hype camps… and ultimately wrong through lazy thinking: people need cognitive shortcuts to dismiss what we have no time to care about because we’re so overloaded with information.
To me, it seems significant these realizations are becoming more widespread as AI gets more widely used. It’s a tool that does a great job connecting disparate information—it basically operates exclusively in the realm of metaphor, of similar “distances” between the vibes of different tokens. It also kills summarizing complex articles or my stream of consciousness ramblings into nice ordered lists with pretty headers. Making order out of chaos.
Someone somewhere who is smarter than me is hopefully asking how to harness the power of AI toward moving us into more liminality, nuance, and complexity, given that instead of 1’s and 0’s it operates on a matrix of relationships between different points in an n dimensional space. It exists in the squishy in-between. It operates more like the universe does, through order arising via the interplay of relationships.
It ought to be better at connecting us than the binary choice between Like/Dislike that was offered by social media, the previous media revolution.
What do you think?
I think that's just human nature. AI won't change that haha. It's not social media or information overload rather those are consequences of how we already were. Mercier and Sperber found this to be true: we're not scientists at heart but lawyers and politicians. We don't seek truth but predictability. We want the universe to be intelligible and not at all surprising. We believe what we want to believe because revisiting preconceptions requires a lot of mental effort. And the brain, for all its virtues, it's very good at tricking us.