It also could be like the Turing test.

There was no grand announcement of passing the Turing test or not. Instead the whole idea has faded in importance.

As the models get better, it wouldn't be shocking to me we get to a point that no one cares if the models are considered "AGI" or not.

We will be chasing some new vaguely defined concept.

AGI is being able to learn from first principles, not being trained on trillions of examples. If you can start with priors and perform a demonstration that is not already a prior then you are intelligent. It’s the difference between the result and understanding the process that produces the result.

Can you elaborate on the process by which you created this definition?

I just applied some stuff that’s been known for a long time in a different context. But let me give you a scenario to think about as a demonstration of how I mean.

Imagine we trained an AI on everything ever written, but the catch is we’ve restricted the training data to the year, let’s say, 400 BCE and earlier (Ignore the fact that most of what was written then is lost to us now and just pretend that’s not issue for our thought experiment) The AI is also programmed to seek new knowledge based off that starting knowledge.

Also pretend that this AI has an oracle it could talk to that would help the AI simulate experiments. So the AI could ask questions and get answers but only in a way that builds ever so slightly off what it already knows.

Making any progress at all in this experiment and discovering new knowledge is what we’re after and “new knowledge” would be defined as some r that’s demonstrated using some p and q as propositions, where r is neither p or q, and r is also correct in terms of 2025 knowledge.

If the AI, with the aid of the knowledge it started with and the help of the oracle to let it ask questions about the world and build off that knowledge, can ever arrive, or exceed, 2025 knowledge then it’s at least generally intelligent and equal to a human. Although the bar could maybe be even less.

It loses, however, if it never advances, gets stuck in a loop, or in some other sense can’t make progress.

This is intelligence: to proceed from things everyone agrees on and ask questions and formulate assertions that depend on propositions holding true, and in the process demonstrate new things that were not already part of common belief.

I don’t know how this experiment could be done in real life with an LLM for example but this is a story version of what I mean in my original comment.

> I just applied some stuff that’s been known for a long time in a different context.

I tried this, and screamed "Goal!!!!" on a golf course when the ball went in the hole. Did not work as expected. Lost my membership.

But maybe I applied it wrong.

Point taken! The question now turns to if I made the same mistake? I’d be grateful if you could demonstrate my mistake for me though, because I don’t see it.

Not at all. I was just implying that the process definition was a bit vague. Next time I'll state it clearly.

Understood. I couldn’t explain everything about the process because it would be too long, but it’s basically Aristotle’s Organon.

Wake me up when there is a formal definition of human sentience that everyone agrees with.