With all this discussion about whether LLMs are “intelligent” or “really understand” it’s getting really obvious that the words “intelligent” and “really understand” aren’t very useful words any more (in this context).
Like, most of us agree about what LLMs can and can’t do and how they do it. So we’re just having fun arguing about how we’d prefer to define “understand?”
On the contrary, it's important to realize that these models have no sort of understanding in any definition of the word. We can always decide to redefine the word "understanding", but that doesn't seem like it would provide any sort of benefit. Perhaps outside of avoiding a difficult conversation about the (huge) gap between ML models and consciousness.
Strongly disagree. Mathematician Gregory Chaitin said "compression is comprehension", and I think this is exactly correct: something is "more understood" if it can be internally represented or externally reproduced with less information.
Based on how transformers are trained to predict the next token, you can view them as powerful compression algorithms [1]. They truly must - and do - understand complex concepts to generate text as they do.
Understanding isn't the full picture for intelligence. Marcus Hutter has argued that AGI requires both compression/comprehension, and being goal directed/having agency. I'd say chatGPT has advanced understanding, but zero agency.
There's the hutter prize which is related to that. I feel like it sort of missed the mark in terms of scale though: the compression techniques needed to compress a relatively "small" and specific dataset like Wikipedia are completely different from the techniques needed to "compress" the entire internet. It's only with the latter that we're seeing something interesting start to happen. The other constraint the hutter prize has is lossless compression, which isn't really conducive to general "learning"
Fair enough. I can see how for a limited domain of understanding, it can be a reasonable analogue for compression. I still don't see the benefit of reducing the domain of a word just to be able to put a new technology on arbitrary scale of "intelligence". I think we should be able to appreciate the power of ChatGPT-like models without lossy compressing of our vocabulary in the process.
Not in any real sense - it just reacts to the prompt, and has no choice in how.
However, if you ductaped a simple goal directed reinforcement learning embodied agent onto a large language model, I think you'd approach something resembling animal-like intelligence. It would have motives, and the means to try to achieve them and adapt to it's environment. There's research in using LLMs for "internal dialog" in robots which is super interesting (https://innermonologue.github.io/)
It's not that we don't understand that this is just a language model.
We do understand that.
The fact that a language model can produce convincing output hints that it is not that different from us.
Imagine something in 5 or 10 years time with 1000 times the power. What will it be like?
I know you think our awareness makes us fundamentally different. But what if it doesn't? What if we're just a machine that likes to anthropomorphize things reflected back on itself?
Ultimately, we won't have to know the answer.
In 10 years AIs like this will be in computer games. Some people will find the results of that disturbing. Is highly accurately similated suffering different to real suffering? The philosophical answer won't matter, the practical answer will be that AI use in games will become regulated. And then you have the first glimmer of AIs getting rights.
How do you explain how it knows the output of the first Python expression in this article, if not by understanding it?
Let alone the fact that’s it’s running in a simulated virtual machine, and the other examples presented in this article, which add layers and layers of complexity on top of that.
I believe one would have to foray into Martin Heidegger for this, understanding not as an epistemic term but as an ontological category: "Understanding is… the original form of the realization of Dasein, being-in-the-world" [1]. And since GPT is not embodied, by definition it is not "under-standing". Heidegger uses again and again this language game of hypokeimenon [2], that which lies underneath, standing under, the subject.
Why should understanding entirely depend on being "embodied"? This is like defining understanding as "a thing humans do" and then insisting that by definition only humans can do it.
The first link then goes on: "One central element of this view is that we are always projecting possibilities onto the world around us. ... To take an example that Samantha Matherne (2019) uses to illustrate Heidegger’s view: when I first apprehend the martini in front of me, I take it as offering a variety of possibilities—to be sipped, to be thrown, to be shaken, to be stirred. If I then take the martini as to be sipped, I am seizing on one of these possibilities and interpreting the martini in light of this specific possibility." GPT on encountering a martini will similarly consider many possibilities like this; you can run it multiple times or configure it or merely ask it to show these different possibilities it thinks can come from a situation. It seems like this definition has a lot more in common with things GPT does rather than being exclusively related to having a body.
"Why should understanding entirely depend on being 'embodied'?"
By asking such a question you are still considering understanding as an epistemic term and not an ontological category. The example given by Ms. Matherne suffers from the same ontic reduction. I will not even try to give an example of "understanding as ontological category", taking the James Cameron way: the (language) technology is not ready yet, maybe never will; perhaps to give such an example we would require a memory transfer technology [1].
"insisting that by definition only humans can do it"
Yes, this is the entire Heideggerian project: humans as Dasein. Beyond tongue-in-cheek, Heidegger is concerned with the history of Beyng (notice capital B and unusual y, in German he spells it as "Seyn", as opposed to "Sein", the Germans capitalize all nouns by default) in which there is a "first beginning", during the Ancient Greek resolvement of metaphysics (even the word, μετά, beyond, φυσικά, physics, that which grows, is used initially only to group some texts by Aristotle written after Physics), and his search for "the other beginning". This is all very obscure. And Heidegger doesn't help it by literally inventing a language without providing a Rosetta stone. In The Event [2] he has quotes like "'Technology' as the basic truth of history qua happenstance" under the title "163. The Saying", make of it what you will. However, just above "162. The demise of metaphysics", he says "[i]f thinking has passed over into erudition [...] [e]ven those who are thoughtless will then recognize how inessential it is". So he can write clearly also. But again, to quote Edward Feigenbaum, "What does he offer us? Phenomenology! That ball of fluff. That cotton candy!" [3].
Again, this all was to give an example of a thinking with no overlap with GPT. As I see it now, departed from Heidegger's view, the problem is how we continue Galileo's project, "measure what can be measured, and make measurable what cannot be", with or without ontological categories.
Like, most of us agree about what LLMs can and can’t do and how they do it. So we’re just having fun arguing about how we’d prefer to define “understand?”