So your solution would be to fine tune the LLM with new knowledge? How do you make sure it preserves all facts and connections/relations and how can you verify during runtime it actually did, and didn't introduce false memories/connections in the process?

I think RAG has a lot to say here. New content / facts go through the embedding process and are then available for query.

I don't generally disagree that a more discrete (not continuous) knowledge base will be another component to augment ai systems. The harder part is how do you build this? (Curate, clean, ETL, query) Not sure a graphdb is the best first choice. Relational DBs can take you pretty far and it is unclear how many 1+N or multi-hop queries you'll need in a robust ai / agent system

I think you are misunderstanding. An embedding places a piece of knowledge in N dimensional space. By using vector distance search you are already getting conceptually similar results.

Semantic difference doesn’t define relationships between semantically disimilar entities in the same way a structured knowledge graph would let you add a new learned relationship. Similarly you can’t necessarily do entity resolution with prurely emebeddings since you’re again just comparing similarity based on the embedding model you’re using rather then the domain or task you’re accomplishing which could differ a lot depending on how generalized the embedding model is vs what you’re doing.

AFAICT most of the "graph" rag implementations discussions, instead of fancy graph queries & or structured knowledge graph, mean:

1. Primary: Inverted index on keywords (= entities). At ingest time, extract entities and reverse index on them. At query time, extract entities and find those related documents, and include next to the vector results as part of the reranking set, or maybe something fancier like a second search based on those.

2. Secondary: Bidrectionally linked summary. At index time, recursively summarize large documents and embed+link the various nested results. At retrieval time, retrieve whatever directly matches, and maybe go up the hierarchy for more.

3. Secondary: Throw everything into the DB - queries, answers, text, chunks - and link them together. As with the others, the retrieval strategy for getting good results generally doesn't leverage this heterogeneous structure and instead end up being pretty simple & direct, e.g., any KV store.

AFAICT, KV stores are really what's being used here to augment the vector search. Scalable text keyword reverse indexing is historically done more on a KV document store like opensearch/elasticsearch, as it doesn't really stress most of the power of a graph engine. Recursive summaries work fine that way too.

Multihop queries and large graph reasoning are cool but aren't really what these are about. Typed knowledge graphs & even fancier reasoning engines (RDF, ...) even less so. These retrieval tasks are so simple that almost DB can work in theory on them -- SQL, KV, Graph, Log, etc. However, as the size grows, their cost/maintenance/perf etc differences show. We do a lot of graph DB + AI work for our dayjob, so I'm more bullish here on graph long-term, but agreed with others, good to be intellectually honest to make real progress on these.

I will say that sometimes you want a very specific definition of a thing or process to get consistent output. Being able to associatively slurp those up as needed is handy.

Not only that, the embedding process can represent relationships and ontology, especially the subtle aspects that are hard to capture in edges or json

Go back to the original word2vec example, how would you put this in neo4j, in generalization?

king - man + woman = queen

Is it not valuable though to differentiate statistically inferred relationships from those that are user-declared?

I would think that a) these are complementary, and b) the potential inaccuracy of the former is much larger than the latter.

Absolutely, however this project says

> Llamaindex was used to add nodes into the graph store based on documents.

So it sounds like they are generating it based on LLM output rather then user-defined. I also wonder how often you need more than a single hop that graphdbs aim to speed up. In an agent system with self-checking and reranking, you're going to be performing multiple queries anyhow

There is also interesting research around embedding graphs that overlaps with these ideas.

I like your answer and a great example of the limitation of knowledge/semantic graphs. Personally, I'd put a knowledge graph on top of the responses to expose it to LLM as an authority & frame of reference. I think it should be an effective form of protection against hallucination and preventing outright incorrect / harmful outputs in contradiction with the facts known by the graph. At least in my experiments.

Topological relationships vs metric relationships. I suppose a great embedding could handle both, but a graph database might help in the tail, where the quality of the embeddings is weaker?

Doesn't a topological space require a metric?

I agree that something that is more like an SQL query, where you have definitive inclusion, will be useful. The harder question is how do you build something like that? How much AI involvement is there in creating the more discrete relation knowledge base.

I dont know if you need a graphdb in particular but there are likely explicit relationships or entities to resolve to eachother that you’d want to add that aren’t known by a general model about your use case. For example if you are personalizing an assistant maybe you need to represent that “John” in the contacts app is the same as “Jdubs” in Instagram and is this person’s husband.