PageIndex Deep Dive: A RAG Architecture That Replaces Vector Search with LLM Reasoning

RAG — retrieval-augmented generation — has practically become a synonym for one specific recipe: throw your documents into a vector database, and search by semantic similarity. But there's a project called PageIndex, from VectifyAI, that pushes back hard on that assumption. Its core argument is deceptively simple: similarity is not the same thing as relevance. Just because a passage *sounds* like your question doesn't mean it actually *answers* it. So instead of using embeddings to find the most look-alike chunk of text, PageIndex lets a language model reason directly about where the answer actually lives.

Let me walk you through how it works, and then contrast it with the very different setup running on this blog.

PageIndex has two phases. The first is building the index. Now, most RAG systems chop a document into chunks and scatter them across embedding space. PageIndex does something closer to how a human reads — it builds a hierarchical tree, basically a table of contents. Picture a financial report. The root node is the whole document. Underneath it sit child nodes: the income statement, the balance sheet, the cash flow statement. And each node carries four things — the section title, the page range it spans, a short LLM-generated summary of what's in it, and pointers to its children. So the income statement node might say "pages five through twenty-two, revenue four-point-two billion, gross margin thirty-eight percent." By default each node caps out around ten pages or twenty thousand tokens, and it can even auto-detect an existing table of contents by scanning the first twenty pages.

The second phase is retrieval — and this is where it gets interesting. There's no vector search at all. Instead, an LLM agent gets three tools. One tool fetches basic document info, like page count. A second fetches the full tree structure — but *only the summaries*, not the actual text. And the third fetches the real content of specific pages you name.

The agent is forced to work in order, like an expert flipping through a book. Say you ask, "What was the 2023 gross margin?" The agent first pulls the tree structure, reads the summaries, and reasons: gross margin? That belongs in the income statement. It sees the income statement lives on pages five through ten. So it requests *only* those pages, reads them, and answers — gross margin was thirty-eight percent, found on page seven. It never touched the balance sheet or the cash flow section. It navigated straight to the answer.

And the payoff is dramatic. On FinanceBench, a financial-document question-answering benchmark, PageIndex hits ninety-eight-point-seven percent accuracy — way ahead of traditional vector RAG. Now, notice *why* that scenario suits it so well: the documents have a fixed, predictable structure, the answers demand precise numbers, and a sloppy chunk split would directly produce a wrong answer. The project's banner is "vectorless, reasoning-based RAG," it's pulling thirty-three thousand stars on GitHub, and it's now extending the same idea from single documents up to an entire corpus.

Okay — so that's PageIndex. Let me switch to what this blog, Engineer News, actually runs, because it's a completely different animal: vector search as the main path, keyword search as the backup, all living on the Cloudflare edge.

The indexing side is refreshingly plain. Take a Markdown article, split it into paragraph-sized chunks on blank lines — capped around a thousand characters each. Embed every chunk with the bge-m3 model into a thousand-and-twenty-four-dimension vector. Store those vectors in Cloudflare Vectorize using cosine similarity, and keep the chunk metadata in a D1 SQLite table. The splitting logic is nothing fancy — it just accumulates paragraphs until adding the next one would blow past the size limit, then starts a fresh chunk.

The query flow goes like this. A user question comes in, gets embedded by the same bge-m3 model, and gets thrown at Vectorize to pull back the top eight matches. Those chunk IDs are used to fetch full content from D1, joined against the posts table for metadata. Then it dedupes — at most one chunk per article — down to five sources total. Now here's the fallback: if vector search comes back empty-handed, it drops to a plain SQLite keyword search, a LIKE query, weighted across title, summary, and content. Finally, everything gets assembled into a prompt and streamed through qwen-14b back to the user.

So how do these two routes really differ? Let me lay it out as a head-to-head.

On index structure: PageIndex builds a hierarchical tree of sections and summaries; this blog uses flat paragraph chunks. On the vector database: PageIndex needs none at all; the blog leans entirely on Cloudflare Vectorize. On the retrieval mechanism: PageIndex uses an LLM agent making tool calls; the blog uses raw cosine similarity. PageIndex has no fallback — reasoning *is* the whole path — whereas the blog falls back to keyword search. PageIndex preserves the document's section hierarchy completely; the blog loses all structure the moment it chunks. PageIndex is built for long documents like financial reports; the blog is tuned for short posts. PageIndex supports full multi-turn conversation and gives you a traceable reasoning path you can audit; the blog is single-turn, and honestly, vector scores aren't very intuitive to explain. And the big tradeoff — cost. PageIndex is expensive, because every navigation step is another round of LLM reasoning. The blog's approach is cheap, running on Workers AI at the edge.

Now, the heart of all this — the claim that deserves real respect — is "similarity is not relevance." When you do vector search, you're really asking, "how semantically close is this text to my query?" But that's not the question you care about. The question you care about is, "can this text actually answer me?"

Here's the killer example. You ask for a company's 2023 EBITDA. Vector search happily recalls every passage that *mentions* EBITDA — the methodology section, historical comparisons, accounting-standard explanations. All semantically close. But the one thing you need is a single line of numbers buried on page twelve of the report. And if that line got split into a different chunk from its surrounding context, vector search just... misses it. PageIndex's reasoning, on the other hand, knows the EBITDA figure only lives in the income statement, and it walks straight there.

But — and this is the crucial caveat — that advantage has a prerequisite: the document needs structure. Financial reports, legal contracts, technical manuals — clear sections, clear hierarchy. A tree index shines there. For short blog posts like the ones on this site, the paragraphs aren't structurally that different from each other, and plain vector similarity is already a perfectly good proxy.

So the takeaways. First: these two architectures aren't competing — they're answering different questions. PageIndex is for long, structured documents where you need exact facts and you can afford GPT-4o-level inference cost and you want to trace which page an answer came from. Vector RAG is for semantic search across big piles of short, structurally-flat documents where you need low latency and cheap edge deployment — blogs, news, notes.

Second: for this blog's actual job — searching short posts — the simple vector approach is a sound, sensible tradeoff. No need to over-engineer it.

And third, the idea worth stealing even if you never adopt the full architecture: PageIndex suggests embedding a combination of *title plus summary plus chunk*, rather than embedding the bare chunk text alone. That small change gives each vector more context to anchor on — and that's a borrowable win for almost any RAG system, including this one.