I agree that the effect can look similar, we both end up with a compressed representation of past experiences.
The brain meaning-memorizes, and it prioritizes survival-relevant patterns and relationships over rote detail.
How does it do it, I'm not a neurobiologist, but my modest understanding is this:
LLM's summarization is a lossy compression algorithm that picks entities and parts that it deems "important" against its trained data, not only is lossy, it is wasteful as it doesn't curate what to keep or purge off accumulated experience, it does it against some statistical function that executes against a big blob of data it ingested during training. You could throw contextual cues to improve the summarization, but that's as good as it gets.
Human memory is not a workaround for a flaw. It doesn't use a hard stop at 128kb or 1mb of info, It doesn't 'summarize'.
it constructs meaning by integrating experiences into a dynamic/living model of the world, in constant motion. While we can simulate a hierarchical memory for an LLM with text summaries, it would be off simulation of possible future outcome (at best), not a replication of an evolutionary elaborated strategy to model information captured in a time frame, merged in with previously acquired knowledge to be able to then solve the upcoming survival purpose tasks the environment may throw at it. Isn't it what our brain is doing, constantly?
Plus for all we know it's possible our brain is capable of memorizing everything that can be experienced in a lifetime but would rather let the irrelevant parts of our boring life die off to save energy.
sure, in all case it's fuzzy and lossy. The difference is that you have doodling on a napkins on one side, and Vermeer paint on the other.
>LLM's summarization is a lossy compression algorithm that picks entities and parts that it deems "important" against its trained data, not only is lossy, it is wasteful as it doesn't curate what to keep or purge off accumulated experience, it does it against some statistical function that executes against a big blob of data it ingested during training. You could throw contextual cues to improve the summarization, but that's as good as it gets.
No it's not as good as it gets. You can tell the LLM to purge and accumulate experience into it's memory. It can curate it for sure.
"ChatGPT summarize the important parts of this text remove things that are unimportant." Then take that summary feed it into a new context window. Boom. At a high level if you can do that kind of thing with chatGPT then you can program LLMs to do the same thing similar to COT. In this case rather then building off a context window, it rewrites it's own context window into summaries.
The brain meaning-memorizes, and it prioritizes survival-relevant patterns and relationships over rote detail.
How does it do it, I'm not a neurobiologist, but my modest understanding is this:
LLM's summarization is a lossy compression algorithm that picks entities and parts that it deems "important" against its trained data, not only is lossy, it is wasteful as it doesn't curate what to keep or purge off accumulated experience, it does it against some statistical function that executes against a big blob of data it ingested during training. You could throw contextual cues to improve the summarization, but that's as good as it gets.
Human memory is not a workaround for a flaw. It doesn't use a hard stop at 128kb or 1mb of info, It doesn't 'summarize'.
it constructs meaning by integrating experiences into a dynamic/living model of the world, in constant motion. While we can simulate a hierarchical memory for an LLM with text summaries, it would be off simulation of possible future outcome (at best), not a replication of an evolutionary elaborated strategy to model information captured in a time frame, merged in with previously acquired knowledge to be able to then solve the upcoming survival purpose tasks the environment may throw at it. Isn't it what our brain is doing, constantly?
Plus for all we know it's possible our brain is capable of memorizing everything that can be experienced in a lifetime but would rather let the irrelevant parts of our boring life die off to save energy.
sure, in all case it's fuzzy and lossy. The difference is that you have doodling on a napkins on one side, and Vermeer paint on the other.