Agent Memory Wants a Wiki, Not a Vector Store

Two things crossed my feed last week that belong in the same post. First, Harrison Chase has been reframing agent memory as a wiki: hyperlinked pages, “wiki as a cache,” pages that scale because you only open the ones you need. Second, a logging study on Hacker News measured what Claude Code actually sends per request. About 33k tokens of system prompt and tool definitions land before your own prompt shows up. OpenCode does the same job with about 7k.

Put those next to each other and you get the real design constraint for agent memory. Context is the scarce resource. The question is not how much you can retrieve, it’s how little you can load while still knowing where everything is.

The default answer since roughly 2023 has been a vector store. Chunk the documents, embed them, cosine-match against the query, inject the top k chunks. I think that default is wrong for most agent setups, and I say that as someone running the alternative every day. My memory layer is a markdown wiki with no embeddings anywhere in the pipeline, and it has quietly become the most reliable part of my system.

Retrieval is two reads#

The vault is plain markdown. An agent does all the ingestion: podcasts, posts and papers get fetched as raw transcripts, distilled into concept and entity pages, and the same agent maintains one special page, the index. Every page in the wiki gets one line there: a link plus a one-line description. Retrieval looks like this:

read wiki/index.md                                # every page, one line each: the cheap layer
read wiki/concepts/memory_as_separate_store.md    # drill down only where the index pointed
read wiki/entities/abridge.md

That’s it. No chunking, no embedding job, no similarity threshold to tune. The index is a couple hundred lines, costs a few thousand tokens to load, and after that the model knows what exists and opens two or three full pages instead of receiving fifteen disconnected chunks. It’s a cache hierarchy: one cheap page that’s always loaded, expensive pages loaded on a miss.

Three invariants keep it honest#

A folder of markdown isn’t memory. What makes this work is what the ingestion agent enforces on every write.

Every claim carries its source. A claim in a concept page is a bullet with a link to the raw transcript and a timestamp anchor:

- Memory baked into model weights is fragile in a fast-moving model
  landscape; decouple memory from the underlying model so model swaps
  don't erase user state.
  [[raw_2026-05-14_latent_space_inside_abridge_the_ai_listening]] @30:55

The raw pages are full transcripts where every line links to the video at that timestamp, so @30:55 resolves to the actual second of audio the claim came from. When the agent answers from memory, it cites. I can check.

Overviews are dated. Every concept page opens the same way:

As of 2026-05-17, "memory as separate store" names the design choice to
persist user/agent memory in a model-independent external store with a
writer/consolidator agent, rather than baking it into model weights.

An embedding can’t tell you when it was true. A dated sentence can, and any model that reads the page inherits that for free.

Nothing gets silently deleted. When a newer source contradicts a claim, the old claim is struck through with a pointer, not removed:

- ~~IPO priced at $150–160 (raised from $115–125 range), 20x
  oversubscribed, $4.8B raise at ~$48B fully-diluted valuation.
  [[raw_2026-05-14_20vc_anthropic_elon_google_cerebras_ramp]] @46:32~~
  (superseded 2026-05-25 by [[raw_2026-05-21_no_priors_cerebras_63b_ipo]])

This is the part a vector store structurally can’t do. Update a chunk and the old belief is gone. Keep both chunks and they come back side by side with nothing marking which one won. The strikethrough keeps the claim, the correction, and the evidence trail in one place, readable by me and by the model.

Legible beats latent#

The usual pitch for embeddings is recall. The thing I actually care about in a memory system is different: I want to read my own memory, and I want to see why the agent believed something.

With the wiki, synthesis happens at write time, when the model has the full source in front of it, not at query time when it’s guessing from fifteen chunks. A vector store stores what documents said. The wiki stores what I currently believe, with receipts. And when retrieval misfires, I can read the index line that misled it and fix it with a text edit. When a vector store misfires, what do you even tune?

It’s the same argument I keep making about picking the least machinery that does the job. An embedding pipeline is a lot of machinery to avoid writing an index.

Where it loses#

Fair is fair. This does not scale to millions of documents. Nothing whose first step is “read the whole index” does. If you’re building search over a corporate document lake, embeddings are the right tool and you should use them.

But agent memory isn’t that shape. My vault is a few hundred pages after months of daily ingestion, and the index still fits comfortably in one read. Personal working knowledge, and honestly most team-level knowledge, is hundreds of pages, not millions.

The real cost is that writes got expensive. Every ingest updates pages, rewrites index lines, checks old claims for supersession. I’ve made peace with that, because that work is the memory consolidation. You pay for understanding either way. I’d rather pay at write time, once, while the source is in context, than at every single query forever.

The takeaway#

Chase’s “wiki as a cache” framing is right, but I’d push it one step further. The wiki isn’t a cache sitting in front of memory. The wiki is the memory, and the index is the synthesis. A vector store remembers what things said. A wiki remembers what you concluded, when you concluded it, and what changed your mind since.

For an agent that’s supposed to act on my behalf, I know which one I want it reading.

I’m a platform/SRE engineer writing about making agentic AI reliable in production. If you’re building a memory layer for your agents and wondering whether you actually need that vector database, get in touch.