I probably don't understand the modern, complex models. But doesn't it basically predict the next token given the context and the better models use more training data and can consider a larger context, and have more parameters to better retain information from the training data etc.
But the fundamental way they operate is the same - predicting the next token given previous tokens. Where/how does reasoning happen here?
Upfront: I think AI is borderline useless for many of the tasks we give it.
But:
1. Do our neurons not just react the same way every time to the same input? A brain is larger than the sum of its parts.
2. They don't reason, but you can somewhat emulate (or pretend to be) reasoning if you feed something back into itself enough times and it pinky promises reasoning is happening
But the fundamental way they operate is the same - predicting the next token given previous tokens. Where/how does reasoning happen here?