Someone told me caching was pointless for their agent because every customer asks a different question.
They were right about the questions.
They were wrong about the cache.
1
29
And the cache stores the plan, never the result.
The plan is replayed against the data layer every time, under that user's identity, and the row filters do their job.
1
A team spent three weeks tuning a RAG pipeline that was broken before the first chunk.
40,000 pages of equipment manuals. The assistant got about a third of its answers wrong.
So they did the reasonable things.
Raised top-k from 5 to 20.
Swapped the embedding model.
Rewrote the system prompt twice.
Added a reranker.
Retrieval got slower and no more correct.
Here is what was actually wrong.
The manuals were mostly tables. Torque specs, part numbers, service intervals.
The parser flattened every table into a run of prose before anything was chunked. By the time the splitter saw the text, "45 Nm" and the bolt it belonged to were in different sentences.
No chunking strategy recovers that.
Chunking happens after extraction. Extraction is the step nobody owns.
Once the parse is right, this is the order I reach for:
1. Fixed-size → the baseline every other strategy has to beat
Cheap, structure-blind, and good enough more often than people admit.
2. Recursive → the sensible default
Paragraphs, then lines, then words. Still blind to meaning.
3. Document-based → when there is a real heading tree
Only as good as your extraction. A flattened PDF has no headings left to split on.
4. Semantic → long prose with topic drift
Starts a new chunk when consecutive sentences stop being similar. Slow to ingest, and the threshold almost never transfers between corpora.
5. Hierarchical → precise matching with context intact
Search the small child, return the large parent. More storage and a more complex index, and on manuals it is usually worth it.
LLM-based and agentic chunking are real options that I reach for rarely. Agentic chunking in particular makes ingestion non-deterministic, so the same document can chunk differently on a re-run and a retrieval bug becomes impossible to reproduce.
Before any of that, two questions about your parser.
Did it keep the tables?
Did it keep the headings?
If either answer is no, fix that first. Everything downstream gets cheaper.
One thing I've learned building agents on enterprise documents:
Retrieval quality is mostly an extraction problem wearing a retrieval costume.
The embedding model is almost never the thing.
What did your parser do to your tables?
#AgenticAI #AIArchitecture #RAG
1
41
Most teams building AI agents give them a service account.
That one decision is why the agent never makes it past security review.
Here's how a request moves through an agent that does.
10 steps, 10 questions.
1
31
Most designs stop identity at the edge.
The agent then queries as a service account, and the only thing separating one user's rows from another's is an instruction in a prompt.
That works on day one. It leaks on day ninety.
1
6





