> ## Content Index
> Fetch the complete content index at: https://www.reasonforoutage.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# The Document Loaded. The Meaning Didn't.
- URL: https://www.reasonforoutage.com/the-document-loaded-the-meaning-didnt/
- Published: 2026-10-19T12:00:00.000Z
- Updated: 2026-10-20T11:48:15.000Z
- Description: Every ingestion check passed. The answers still mixed up two vendors' instructions.
- Author: Christy Steyn
- Tags: Ingestion, Architecture

The dashboard was green. Chunk sizes looked normal. Documents were being detected. Every citation pointed back to a real source.

And the system kept handing out one vendor's procedure with another vendor's prerequisites.

This is a post-mortem about that, and about a second ingestion failure that looked like the opposite problem. Both started the same way: the words arrived and the meaning didn't.

## Symptom

Answers about operational procedures were sometimes blended. A question about one platform could come back with steps that belonged to a different one. The answer read fine. The citation resolved. You would only catch it if you already knew the procedure.

That is the worst kind of wrong, because it passes a quick read.

## What the source looked like

The source was a wiki page built with tabs, one tab per vendor. If you opened it, you saw clean sections. Vendor A here, vendor B there.

The processing path did not see tabs. It saw text. My best guess is that the tab titles lived in markup the parser skipped, but I have not confirmed that against the page itself, so treat it as a guess.

What I can say for sure is what happened downstream: the vendor boundary was gone by the time chunks were written. Retrieval could find the right page and still hand back the wrong piece of it.

## Why nothing caught it

Because nothing was broken, in the way my checks understood broken.

The parser did not crash. No document was dropped. Chunk sizes were plausible. Citations resolved. Each of those checks answers "did the pipeline finish?" None of them answers "did the meaning survive?"

That gap is the blind spot. A completed ingestion job told me far less than I thought it did.

## What I did, and what I haven't finished

I had two ideas. Turn the tab boundaries into real headings before parsing. And compare a PDF export against the HTML path to see which one keeps the vendor split.

I have not finished that comparison, so I am not telling you PDF wins. What I did build was retrieval that finds small, precise chunks and returns the larger parent for the model to read. Retrieve small, read big. It only helps if the parent still belongs to the right section, which is the exact thing that was going wrong.

## Then it failed the other way

Later I moved a template from the platform onto a different machine and pointed it at a fresh pile of markdown documents. This time sections were not being merged across a boundary. They were barely being assembled at all.

The numbers: 734 chunks, median length 62 characters. 498 of them were under 100 characters. One chunk was just the heading of a weekly ritual, with nothing under it.

The cause was dull. The merge step only ran when the source was marked PDF or HTML. Markdown arrived with no source format at all, so it skipped the merge and sent raw paragraph blocks straight on.

I changed the step to let files with a missing format, markdown, and plain text through. After re-indexing: 61 chunks, median length 1,071 characters.

Those are numbers from one test corpus. They show the shape of the chunks changed. They do not prove the answers got better, and 1,071 is not a magic chunk size.

The point is where the fix went. A bigger model would not have made a skipped step run.

## A smaller version of the same mistake

One more, because it rhymes. A query filtered on a field I thought belonged to wiki pages. It actually belonged to attachment records, and the page records used a different source identifier. The filter looked right in code and matched nothing.

The code, the stored records and the query each had a slightly different idea of the schema. Nobody was wrong in isolation.

## What I do now

I stopped asking whether ingestion finished. I ask what has to survive, and then I check it.

- **Which source it came from, and which version.** I can follow a chunk back to it.
- **Where in the document it sat.** If two platforms share a page, the chunk has to say which one it belongs to.
- **Its warnings and prerequisites.** An instruction without its condition can mean something else.
- **Table headers and step order.** A value is useless without its column, a procedure without its sequence.
- **Who is allowed to see it.** Matching a query is not permission.

Then I take the ugliest real document I have and follow one fact that changes the answer, such as a platform or a prerequisite, through four stages: the source, the parsed blocks, the stored chunks, and the text the model finally sees. The first stage where the fact goes missing is where the bug lives.

If it is gone before retrieval starts, no amount of retrieval tuning will bring it back. An embedding can only represent the text you give it. It cannot recover a tab label that a person saw and your pipeline never did.

## The decision for you

Before you touch the retriever, read your chunks. Pick the document your system answers worst, and do the four-stage walk on a single fact.

Chunk counts and length charts will tell you something is off. Reading the examples tells you what.

*I write post-mortems for agentic systems at Reason for Outage: real failures, traced all the way down. Subscribe if you want the next one.*