Why we dropped RAG for Fireside's spreadsheets
Fireside Ventures wanted plain-English answers from years of portfolio financials. RAG and hybrid search kept losing the thread. A folder of index.md files, a short CLAUDE.md and a small cache did not.

The ask
Fireside is a consumer VC fund with about $650M under management. Every portfolio company sends them monthly MIS reports, annual plans, board decks and more. All of it sits in Box.
An associate's question usually sounds simple. "How did marketplace margins move for this brand over the last four quarters?" But answering it means finding the right company folder, the right month's file, the right sheet and the right row. Then doing it again for the next month, and the next.
They wanted to ask the question and get the answer, with a citation they could check.
Why the spreadsheets fought back
Three things made this hard.
- No shared layout. Every company builds its reports its own way. Different sheet names, different row labels, different column orders. And Fireside can't ask 70 founders to change how they report.
- The layout changes over time. Even one company's file doesn't stay still. Below is one brand's monthly MIS over a single year. The sheet changed shape three times, and the channel columns kept growing as the brand started selling in new places.
- A number needs its neighbours. A cell that says 4.2 means nothing alone. Its meaning lives in the row label on the far left, the column header near the top, and the month in the first row. In a 10,000-row sheet, those can be very far apart.
Attempt one: plain RAG
We started where everyone starts. Cut the spreadsheets into chunks, embed them, and fetch the closest chunks for each question.
It fell apart on the questions that mattered. Chunking by rows cut values off from their headers. Chunking by sheet made chunks too big and too vague. And a rule that worked for one company's file broke on the next one, because the next one was a different shape.
The worst failures were questions across dates. "Compare March to September" needs the same row from two different files. RAG would bring back the right words from the wrong month, and the model would answer confidently anyway.
Attempt two: Typesense hybrid search
Next we tried Typesense, which is famously good at hybrid retrieval. It mixes keyword search with vector search in one query, so exact labels like "CM 2" and fuzzy meaning both count.
It did find better chunks. But a better chunk is still a chunk. If the header was never in it, no ranking can put it back. The cross-date questions still failed.
That was the real lesson. Search quality wasn't the problem. Cutting the data up before the model saw it was.
What worked: give Claude a map
So we stopped chunking. Instead, we let Claude find the right file the way an analyst would, by walking the folders, and then read the whole sheet with code.
It has three parts.
- An
index.mdin every big folder. A short note on what the folder holds: its type (fund, company, shared folder), a one-line summary, tags, which sub-folders have their own index, and links to related folders. Claude follows these pointers down. It never lists folders or searches Box blindly. - A
CLAUDE.mdthat sets the rules. Start at the root index. Follow the pointers. Only open a spreadsheet once you know it's the right one. Find columns from the header row, never by position, because they move. Always cite the file, sheet and cell. - A cache of popular files. A JSON file on disk lists the files people ask about most, with their path, Box ID and what's inside. Claude checks it first. A hit skips the folder walk completely.
A company's index looks roughly like this:
---
type: company
summary: Fund III beauty brand. Monthly MIS from FY25, AOP, board decks.
tags: [fund-iii, beauty, d2c, mis, aop]
child_indexes:
- index_path: MIS/index.md
index_file_id: "1234567890"
links:
- path: Fund III Portfolio/Other Name/index.md
relation: same_entity
---
The links matter too. Some companies sit in Box under a legal name and get asked about by brand name. A same_entity link joins the two, so Claude doesn't miss half the history. Once Claude has the right file, it reads it through the Box MCP and computes with real code, so the header, the row label and the month all stay attached to the number.
Testing it with the people who ask
We didn't write the test questions ourselves. Fireside's associates and investment analysts sat with us and gave us 30 real questions they actually ask. For each one, they went through the data by hand and wrote down the correct answer and the exact citation.
That set became the bar. Every change to the indexes or to CLAUDE.md got run against all 30, and checked against both the answer and the citation. The cross-date questions, the ones RAG kept missing, were the ones we watched most closely.
From the first RAG attempt to a verified, delivered system took about a month and a half.
Getting a whole fund onto AI
Fireside wasn't an AI-first team when they came to us. Most of this digging was done by hand, in Box and Excel. So part of the job wasn't code at all. It was convincing the CTO and the lead investment analysts of one idea: today, everyone on the team should have their own AI assistant.
This was in early 2026, before Opus 5.5 was even out. They agreed, and took a Claude Enterprise plan. Everyone on the team got their own Claude account.
The enterprise plan mattered for one reason above the rest: you can share things across the whole organization.
- One shared skill. We packaged the Box skill with its instructions, the same rules as
CLAUDE.mdplus notes on how each company lays out its files, and shared it with everyone. Nobody had to copy a prompt or set anything up. - One shared connector. We connected the Box MCP for the whole organization. Each person signs in to Box once, and Claude can read the folders they can. No more downloading files to paste into a chat.
- Fix it once, everyone gets it. When we improved the skill or an index, the whole team got the change the same day. Nobody was stuck on an old copy.
So an analyst doesn't open a special tool. They open Claude, ask their question, and Claude uses the shared skill and connector to walk the index and read the file.
What I took away
- Don't chunk data that has no shape. Give the model a map and let it open the whole file.
- Better search doesn't fix missing context. We could have tuned retrieval for months. The fix was to stop retrieving fragments.
- Plain markdown is a fine index. An analyst can open an
index.md, read it, and fix it. Try doing that with an embedding. - Let the people who ask the questions write the test. Their 30 questions told us more than any benchmark would have.
- Rollout is part of the work. A shared skill and a shared connector turned one pilot into a tool the whole team uses every day.