You're the reason why I had to write this post. Thank you. (Now, please, specify in technical terms how consciousness emerges from brain activity or how intelligence emerges from chemistry.)
You're the reason why I had to write this post. Thank you. (Now, please, specify in technical terms how consciousness emerges from brain activity or how intelligence emerges from chemistry.)
What has consciousness got to do with it? And I am not persuaded that there is a valid analogy to be made between AI/math and human intelligence/chemistry.
Consider a pretrained generative large language model. It's internal state, i.e. the mathematical operations that it will perform on any given input, are fixed. From the given input, it generates a corresponding deterministic output comprising a vector of next-token probabilities for every token in its vocabulary. One of these tokens is chosen, added to the input sequence, and then the process is repeated until some stopping condition is satisfied.
If the most probable token is chosen every time (equivalent, in most configurations, to setting the 'temperature' parameter to zero) then the entire output from a given input is entirely deterministic, derived from the frozen model state and the input. For example, running Mistral-Nemo 12B locally under Ollama, if I use the system prompt "You are a novelist. Take the text provided by the user and continue the story.", set temperature to zero, and provide the user input "It is a truth universally acknowledged...", the model reliably and repeatably reproduces the remainder of the first sentence of Pride and Prejudice, followed by a rather dull summary of the novel's romantic narrative of Mr Darcy and Elizabeth Bennet.
As I turn up the temperature, the output ceases to be repeatable, because each further token is being chosen increasingly at random. But even so, the model generates variations on the characters and plot of Pride and Prejudice. Eventually (at around a temperature of 1.1 in my experiments, in which I have also effectively maxed out top_k and top_p so that all tokens are available in principle) the model transitions rapidly from variations on the plot of the novel in perfect English to complete gibberish in a mixture of languages!
This is a rather disappointing form of 'intelligence'. Without added (pseudo)randomness it is not creative at all. With a little randomness it becomes more creative, but still can't go anywhere with 'It is a truth universally acknowledged...' other than Pride and Prejudice. Why not? There are surely lots of universally acknowledged truths. It is a truth universally acknowledged that a zombie in possession of brains must be in want of more brains. (Yes, that's the opening sentence of the parody novel Pride and Prejudice and Zombies, written by a human.) And with additional randomness it never starts to diverge creatively from Pride and Prejudice; it just suddenly loses the plot completely.
A reductionist might argue here that perhaps human intelligence and creativity is just the result of natural randomness. After all, chemical reactions are affected by the environment in which they occur, which is inevitably uncontrolled and noisy. And even if it weren't, perhaps we could fall back on quantum uncertainty as our source of randomness. But even if that's true, it's hardly the same thing as our generative AI model. The randomness in nature is everywhere, not just in a feedback loop that selects the next 'idea token'. There's no useful or meaningful analogy to be drawn.
My take is that our current generative models are interesting and they are useful. I use them all the time. By scaling them up, and wrapping them in clever applications and prompting techniques we can make them more interesting and useful... up to a point at least (which we seem not yet to have reached). But I think they're a dead end as far as a path to AGI is concerned.
You're the reason why I had to write this post. Thank you. (Now, please, specify in technical terms how consciousness emerges from brain activity or how intelligence emerges from chemistry.)
What has consciousness got to do with it? And I am not persuaded that there is a valid analogy to be made between AI/math and human intelligence/chemistry.
Consider a pretrained generative large language model. It's internal state, i.e. the mathematical operations that it will perform on any given input, are fixed. From the given input, it generates a corresponding deterministic output comprising a vector of next-token probabilities for every token in its vocabulary. One of these tokens is chosen, added to the input sequence, and then the process is repeated until some stopping condition is satisfied.
If the most probable token is chosen every time (equivalent, in most configurations, to setting the 'temperature' parameter to zero) then the entire output from a given input is entirely deterministic, derived from the frozen model state and the input. For example, running Mistral-Nemo 12B locally under Ollama, if I use the system prompt "You are a novelist. Take the text provided by the user and continue the story.", set temperature to zero, and provide the user input "It is a truth universally acknowledged...", the model reliably and repeatably reproduces the remainder of the first sentence of Pride and Prejudice, followed by a rather dull summary of the novel's romantic narrative of Mr Darcy and Elizabeth Bennet.
As I turn up the temperature, the output ceases to be repeatable, because each further token is being chosen increasingly at random. But even so, the model generates variations on the characters and plot of Pride and Prejudice. Eventually (at around a temperature of 1.1 in my experiments, in which I have also effectively maxed out top_k and top_p so that all tokens are available in principle) the model transitions rapidly from variations on the plot of the novel in perfect English to complete gibberish in a mixture of languages!
This is a rather disappointing form of 'intelligence'. Without added (pseudo)randomness it is not creative at all. With a little randomness it becomes more creative, but still can't go anywhere with 'It is a truth universally acknowledged...' other than Pride and Prejudice. Why not? There are surely lots of universally acknowledged truths. It is a truth universally acknowledged that a zombie in possession of brains must be in want of more brains. (Yes, that's the opening sentence of the parody novel Pride and Prejudice and Zombies, written by a human.) And with additional randomness it never starts to diverge creatively from Pride and Prejudice; it just suddenly loses the plot completely.
A reductionist might argue here that perhaps human intelligence and creativity is just the result of natural randomness. After all, chemical reactions are affected by the environment in which they occur, which is inevitably uncontrolled and noisy. And even if it weren't, perhaps we could fall back on quantum uncertainty as our source of randomness. But even if that's true, it's hardly the same thing as our generative AI model. The randomness in nature is everywhere, not just in a feedback loop that selects the next 'idea token'. There's no useful or meaningful analogy to be drawn.
My take is that our current generative models are interesting and they are useful. I use them all the time. By scaling them up, and wrapping them in clever applications and prompting techniques we can make them more interesting and useful... up to a point at least (which we seem not yet to have reached). But I think they're a dead end as far as a path to AGI is concerned.