Will take a bit of time before AI can consistently beat us on coding/proofs but the raw ingredients imo are there. As someone who was skeptical of AGI via just scaling things up even after GPT-3, what convinced me was the chain of thought prompting paper. That shows the LLM can pick up on abstract thought and reasoning patterns that humans use. Only a matter of time before it picks up on all of our reasoning patterns (or maybe it already has and is just waiting to be prompted properly...), is hooked up to a good memory system so it's not limited by the context window and then we can watch it go brrrr
It can still make stupid mistakes in reasoning but I don't think that's fundamentally unsolvable in the current paradigm
It's definitely not just copying verbatim. If you mean it's emulating the reasoning pattern it sees in the training data well...don't humans do that as well to get answers to novel problems?
We don't know all the different ways humans arrive at answers to novel problems.
And while these LLMs aren't literally just copying verbatim, they are literally just token selection machines with sophisticated statistical weighting algorithms biased heavily towards their training sets. That isn't to say they are overfitted, but the sheer scale/breadth gives the appearance of generalization without the substance of it.
Here's an argument that GPT does actually build an internal representation of the game Othello, it's not just token selection: https://thegradient.pub/othello/
Keep in mind that the Othello example is model specifically trained on only Othello games. I haven’t seen any claims that general purpose models like GPT-4 have internal representations of complex abstract structures like this.
Why wouldn't they? Text-moves of Othello games are presumably a subset of the training data for a general LLM. If anything the general LLM has the chance to derive more robust internal world representations given similarly laid out board games.
It is also not surprising that if you force a system to succinctly encode input-output relationships, eventually it discovers the underlying generating process or its equivalent as implied by Kolmogorov complexity theory. Language is just a convenient encoding for inputs and outputs, not fundamental. So yes it is regurgitating statistics, but statistics are non-random because of some non-trivial underlying process, always, and if you can regurgitate those statistics consistently you're guaranteed to have learned a representation of the process. There is no difference and biological systems aren't any different.
This morning I asked GPT-4 to play duck chess with me. Duck chess is a very simple variant of chess with a duck piece (that acts like an impassable brick) that each player moves to an empty square after their normal move. [I gave GPT-4 a more thorough and formal explanation of the rules of course.]
To a human, board state in chess and in duck chess is very simple. It’s chess, but with one square that’s blocked off. Similarly, even a beginner human chess player can understand the basics of duck chess strategy (block your opponent’s development in the beginning, block your opponent’s attacks with the duck to free up your other pieces, etc.).
GPT-4 fell apart, often failing to make legal moves, and never once making a strategically coherent duck placement. To me this suggests that it does not have an internal representation of the 64 squares of the board at all. Even if you set aside the strategic aspect, the only requirement for a duck move to be legal is that you place it on an empty square, which it cannot consistently do, even at the very beginning of the game (it like to place the duck on d7 as black after 1. …e5, even when its own pawn is there).
It is a matter of degree. GPT-4 may, for various reasons some of which are artificial handicaps, have only a weak grasp of a board representation now. But if it has any such representation at all, that's already a different story than if it did not. I think all evidence points this way, even from other networks, e.g. image classification networks that learn common vision filters. It's a pretty general phenomenon.
The statistical distribution of historical chess games is a approximate statistical model of an actual model of chess.
It's "internal abstract representation" isnt a representation; it's an implicit statistical distribution across historical cases.
Consider the difference between an actual model of a circle (eg., radius + geometry) and a statistical model over 1 billion circles.
In the former case a person with the actual model can say, for any circle, what it's area is. In the latter case, the further you get outside the billion samples, the worse the area will report. And even within them, it'll often be a little off.
Statistical models are just associations in cases. They're good approximations of representational models for some engineering purposes; they're often also bad and unsafe.
It's not some kind of first order statistical gibberish.
It exhibits internal abstract modeling.
It's a bit silly to argue against it at this time.
To produce answers with quality we see it'd have to use orders of magnitude more memory than it actually does.
It's also easy to test yourself.
Simple way is to create some role playing scenario with multiple characters when same thing is seen differently by different actors at different time and probe it with questions (ie. somebody puts X into bag labelled Y, other person doesn't see it and asking what different actors think is in the bag at specific time in the scenario etc).
Or ask for some crazy analogy.
Why am I even saying it, just ask it to give you list of examples how to probe LLM to discover if it creates abstract internal models or not - it'll surely give you a good list.
It seems like chain-of-thought will work pretty well when backtracking isn't needed. It can look up or guess the first step, and that gives it enough info to look up or guess the second, and so on.
(This can be helpful for people too.)
If it goes off track it might have trouble recovering, though.
(And that's sometimes true of people too.)
I wonder if LoRA fine-tuning could be used to help it detect when it gets stuck, backtrack, and try another approach? It worked pretty well for training it to follow instructions.
For now, it seems like it's up to the person chatting to see that it's going the wrong way.
Man America is really such a bore: VC, Military, solving problems, weapons, war, getting it done quicker, “freedom”, is there anything else to all this, to life ?
I used to be one of those “why do people always rag on American culture?” types, but I’m getting it.
It makes me laugh how we want to automate everything without the slightest idea of what we’ll be doing once it’s all automated ? Is that the point where the USA figures out we already had a lot of good things to do ? Ha, we’ll invent the matrix and plug ourselves in.
Sorry it’s not personal but it just seems like a never ending grind instilled with the same themes over and over. Now we have “hustle culture with no job prospects due to automation”, cool plan.
Well this approach brings us also modern medicine, so we are not dying in the pool of mud from every trivially treatable malaise. We break atoms, we can almost reach the stars which is probably the only long term way to preserve mankind.
You can't have one without the other. Look at any society in history, either push forward or eventual downfall and ending up as small history lesson. Some say its human nature, some say its nature's nature.
I’m sorry but modern medicine and technologic / scientific advancements happen without the modern American psyche.
If “America” wasn’t a thing , we’d all be completely fine.
In my opinion America is pushing us to “need” to be a multi planetary species. Not the other way around. We’re taking greater risks for the growing need for monetary reward it’s unavoidable.
As others have put it, the Earth is the greatest starship we’ll ever know. We know that the trajectory we’re on will require a backup.
It can still make stupid mistakes in reasoning but I don't think that's fundamentally unsolvable in the current paradigm