Beyond RAG: A Deep Dive into Andrej Karpathy's LLM Wiki Pattern

RAG has been the industry's go-to answer for years. You've probably heard the recipe a hundred times: take your documents, chop them into chunks, turn those chunks into vectors, and when a question comes in, find the chunks that look most similar and stuff them into the model's context. And honestly? It works. It solves the very real problem that a language model doesn't know anything about your private data.

But it leaves another problem completely untouched. With RAG, knowledge is static. Every single query starts from a blank slate. There are no links between your documents, and here's the kicker—the system never gets any smarter, no matter how much you use it. You could ask it the world's most brilliant question today, get a fantastic answer, and tomorrow that answer is just... gone. Vanished. Next time, it recomputes everything from scratch.

In April 2026, Andrej Karpathy published a GitHub Gist proposing a different direction entirely. He calls it the LLM Wiki. And the core idea is this: instead of having the model passively answer each question one at a time, you have it actively build and continuously maintain a structured knowledge base. This isn't a tweak to RAG. It's a different answer to the question of what a knowledge system should even be.

So let's talk about where traditional RAG hits its limits. Picture the standard pipeline: raw documents get chunked, embedded into vectors, dropped into a vector database. A user query comes in, it gets embedded too, you run a similarity search, pull back the relevant chunks, and feed them to the model. Clean. But notice—every query is an island. The vector database only remembers which pieces of text resemble your query. It has no memory of how those pieces relate to each other.

That creates a few concrete headaches. First, cross-document reasoning is weak. Ask something like, "How does this error connect to that incident three months ago?" RAG can dig up both relevant pieces individually, but stitching them together and reasoning across them? That's where it stumbles. Second, there's no compounding—that great answer from today doesn't stick around. Third, chunking itself fragments meaning. Slice a long document into little 512-token pieces and a lot of context just evaporates at the seams. Vector similarity finds text that's *literally* similar, which isn't always the text that's *logically* most relevant. And finally, it's sensitive to scale. RAG is perfectly happy with a few hundred documents. Push it into the thousands, and recall starts dropping while noise creeps up.

Now here's Karpathy's architecture, and it has three layers. Think of them as a stack.

At the bottom you've got Raw Sources—the immutable fact layer. Your original articles, papers, conversation logs, code. The model only ever reads from here. It never writes back. These are your ground truth.

In the middle sits The Wiki—the knowledge layer. This is where the magic happens. After reading the raw sources, the model actively generates and maintains a set of markdown pages. It creates an entry for each important concept, writes summaries that span across documents, and explicitly marks how concepts relate to one another. And crucially, these pages keep updating as new data flows in. They're alive, not frozen.

And on top, governing everything, is the Schema. The schema defines how the wiki grows: what types of pages exist, what format each one uses, and when a given page should get updated. It's the skeleton that holds the whole thing together.

Let's get into how it actually operates, because there are three core operations and they're each interesting.

The first is Ingest. When new data arrives, the model doesn't just chunk it and dump it into a database. It stops and asks real questions: Which existing wiki pages does this new information affect? Do I need to create new concept entries? Does any of this contradict what I already wrote down? So it reads the relevant existing pages, looks at the current state of knowledge, and then makes a decision—add a page, update a page, or flag a conflict—before writing back to the wiki. The point is, you're accumulating *relationships* between pieces of knowledge, not just piling up more data.

The second operation is Query. When you ask a question, the model reads wiki pages—not raw document chunks. And because those pages are already curated, distilled knowledge, the quality of the context is dramatically higher than RAG's scattered fragments. For a question that needs reasoning across concepts, you can first consult the wiki's index to find the right pages, then read deeper into them.

The third is Lint. And I love this one. It's basically a health check you run periodically. The model scans the whole wiki looking for problems: content that's gone stale because a newer source came in but the page never got updated, orphan pages that nothing else links to, and spots where a concept gets defined inconsistently in two different places. It lets the knowledge base heal itself. Your wiki does its own housekeeping.

So how does this really stack up against RAG, head to head? Let me walk through the differences. In RAG, knowledge takes the form of raw chunks plus vectors; in the LLM Wiki, it's curated markdown pages. Cross-document relationships in RAG are only implied through vector similarity; in the wiki they're explicit—actual cross-references and a concept index. Compounding effect? RAG has none, it recomputes every time; the wiki genuinely grows over time. Updating in RAG means re-embedding the whole document; in the wiki you only touch the pages that were actually affected.

Now, the wiki isn't free. Build cost in RAG is low—you just embed and you're done. The wiki is expensive, because an LLM has to run on every ingest. Maintenance is also heavier—you have to design that schema. And the sweet spots differ: RAG is great from tens to a few hundred documents, while the wiki really starts to shine at hundreds and up, where the compounding actually pays off.

And there's a real data point here. At around 100 articles, roughly 400,000 words, Karpathy's wiki clearly beat an equivalent RAG setup on both accuracy *and* speed. The reason comes down to one phrase: wiki pages are digested knowledge, not fragments of raw data.

So when should you actually reach for this? It's a strong fit when your knowledge keeps accumulating—a blog, research notes, evolving technical docs—and you genuinely want the system to get smarter the more you feed it. It's great when you need cross-document reasoning, like "how has the thinking on this problem shifted over time?" It makes sense once you've blown past a few hundred documents and RAG's recall has started letting you down. And it fits if you've got the resources to design a schema and eat the LLM cost on every ingest.

It's a poor fit in the opposite cases. If your documents are static and barely change, a one-shot RAG is plenty. If you need to query the very latest data in real time, remember the wiki's ingest has latency—it's not instant. If you're resource-constrained and can't afford an LLM run every time data lands, skip it. And if your knowledge base is small, just a few dozen documents, RAG already does the job fine.

Here's a practical rule of thumb I really like: if you keep asking your knowledge system the same kind of question but you find yourself re-explaining all the background every single time—that's your signal. That's when the LLM Wiki is worth a serious look.

The community jumped on this fast. Within a short time there were over fifty implementations. There's OmegaWiki, a full three-operation pipeline built in Python with the Claude API. There's SwarmVault, a multi-agent take where different agents own different subject domains. And there's WeKnora, which wires Obsidian in as the storage backend. The big open challenge right now is schema design—there's no standard for what a wiki page should look like, different domains need different structures, and a lot of it still comes down to hand-crafting. Karpathy hasn't even fully published his own schema yet.

So let me leave you with the core takeaways. First, the fundamental shift: RAG says "go look it up when you have a question," while the LLM Wiki says "keep the knowledge curated continuously, so when the question comes, the answer's already waiting." That's the whole philosophy in one line. Second, the wiki's real superpower is compounding—it's built for personal knowledge bases, technical docs, and long-term research notes that grow over time, whereas RAG stays the lighter, simpler choice for real-time lookups and one-off questions. And third—and this might be the most useful thing to walk away with—these two aren't enemies. The most realistic future might be using the LLM Wiki as a preprocessing layer that sits in front of RAG: let the model curate the knowledge first, then let vector search pinpoint the exact spot. Curate, then retrieve. You get the best of both.