Don't people get tired of having this same "debate" on every post about LLMs? And I scare quote debate because the naysayers never support their strong claims beyond the most superficial of responses. It's all just so tiring at this point.
"conceptual understanding of something" is just another way of saying "the relationship between concepts", which is exactly what transformer models use.
*EDIT* To elaborate, how can you define anything in isolation of every other concept/thing? You can't. Things are only defined by their relationships to each other, which is exactly the same thing transformer models do.
No, it isn't. "Conceptual understanding" is a deep comprehension of a particular concept. It is grasping its meaning, significance, applications, and boundaries. It involves knowing not just what something is definitionally, but understanding how it works, why it matters, and how it connects to other ideas.
"The relationship between concepts," is focusing specifically on how different ideas connect, overlap, contradict, or complement each other. It's more about the network or system of connections rather than deep comprehension of individual concepts.
Understanding relationships between concepts is part of conceptual understanding, sure. But conceptual understanding is broader - it includes both mastery of individual concepts and awareness of their relationships to other concepts.
> It is grasping its meaning, significance, applications, and boundaries
To define "thinking" by using words like "meaning", "understanding", or "comprehension" just moves the need for definition further up the abstraction ladder. It doesn't help to define what "thinking" is in any quantifiable way.
To play along, could you define "meaning" or "understanding" in a way that doesn't resort to ghost-talk or just move the definition even further up the abstraction ladder? They are both subjective terms that describe how humans feel, not well defined words that describe objective reality in some way.
To use a more quantifiable metric we could look at something like Humanity's Last Exam. OpenAI's o3 scores something like 20% (a feat which few humans could accomplish). To put that in perspective, consider that fifty four percent of Americans now read below the sixth grade level. Like it or not the machines are "smarter" than the majority of humans and have deeper "understanding" in most of the objective ways we've thought of to measure it. Subjective feelings aside, it's tough to argue that the machines aren't conscious if we're going to accept that our fellow citizens are.
I am not sure that you can make that absolute statement. Reasoning is subdivided into types, and one of those types is inductive reasoning.
> Inductive reasoning refers to a variety of methods of reasoning in which the conclusion of an argument is supported not with deductive certainty, but with some degree of probability. Unlike deductive reasoning (such as mathematical induction), where the conclusion is certain, given the premises are correct, inductive reasoning produces conclusions that are at best probable, given the evidence provided.
Doesn't predicting the next token qualify as doing just that?
An LLM is not a Markov process. They are fundamentally different. An LLM conditions the next token prediction on the entire context window (via the attention mechanism), not just the previous token. Besides the token history window it also maintains a cache of neural activations which is updated at every step.
Otherwise you could use the same reasoning to argue that a human is a Markov process, which is absurd, but vacuously true if "state" means the quantum level configuration of every atom in the body.
To push back on this, a somewhat recent Linus Torvalds ~quote:
"I don't think that 'just predicting the next word' is the insult that people think it is, it's mostly what we all do."
If we break our lives down into the different types of reasoning, and what we mostly do day-to-day, this rings very true to me.
I currently believe that our brains generally operate as very efficient inference machines. Sometimes we slow down to think things through, but for example, when in the ideal "flow state" it's some kind of distilled efficient inference. Isn't it? This is very hard for me to deny at this time.
___
edit:
4o appears to agree with both of you, more than it does with me.
I may be wrong, but it seems to me this also is a case of improper use of words.
Those LLMs neither agree nor disagree. They do not understand. They produce output, and we read that output and we ourselves consider the output to be something, or something else.
All an LLM does is produce output. There's no conceptual understanding behind it, and so there is no agreement, or disagreement.
> All an LLM does is produce output. There's no conceptual understanding behind it, and so there is no agreement, or disagreement.
I think that I agree. However, even on HN, what percentage of human comments are simply some really basic inference, aka output/"reddit"/etc... and those are humans.
I am not trying to elevate LLMs to some form of higher intelligence, my only point is that most of the time, we are not all that much better. Even the 0.000001% best of us fall into these habits sometimes. [0]
I currently believe that modern LLM architecture will likely not lead to AGI/ASI. However, even without that, they could do a lot.
LLMs learn high-dimensional representations that capture conceptual relationships in their training data. They manipulate those representations in ways that approximate human reasoning.
System 1 is rapid, uses heuristics to make quick judgements. Not rigorous. System 1 is the default mode.
System 2 is slow deliberate reasoning, energy intensive, and even humans get that wrong.
LLMs often use something like System 1 pattern matching, get the answer wrong initially, then can be prodded into trying again with a System 2 approach (chain of thought).
Sorry to join the pile-on, but can I just ask: In what way does a brain think that an Ai does not? And does the distinction apply from human brains down to fruit flies? Is it a property of embodiment? (I have suspected for years that consciousness isn't just emergent but specifically that it is NOTHING besides that. It's all about scale and large models are just starting climb the ladder. The ladder does not necessarily go up the same way as embodied thought though.)
How can you be so sure? How do you know that our brains don't work like transformers too, except for having the advantage of having more types of sensory data? How can you settle this debate without defining what "thinking" and "reasoning" is and how what LLMs do is not similar to what a kindergarten level kid may be capable of? I think we all agree kindergarten kids can think and reason, don't we?
We should also not be calling a pointing device "a mouse" because it's not a small rodent, there aren't any actual windows inside a computer, and I haven't seen anyone balancing their laptop trying to surf the web.
Also smartphones are not actually smart and are only barely phones.
I'm finding laymen are thinking AI is reasoning, because the term makes it look like this is what it is.
The potential confusion of terms such as mouse/windows/surfing is not the same as calling LLM AI, and then going on to say it is "thinking" and "reasoning".
