docs is a list of loaded Document objects, as produced in RAG with Deep Agents.
Balance chunk size and overlap
Chunk size trades precision against context, and there is no universal best value. It depends on your documents, your embedding model, and the questions you expect.- Small chunks produce focused embeddings and precise matches, but a single chunk may lack the surrounding context the model needs to answer.
- Large chunks carry more context per chunk, but their embeddings blur together multiple topics, which makes them harder to match and spends more tokens per retrieved chunk.
Treat these as starting points and confirm them by evaluating your chunking.
Start with recursive character splitting
Start with RecursiveCharacterTextSplitter. It splits on a descending list of separators (paragraphs, then sentences, then words), which keeps semantically related text together whenever it fits within the target size. This is the recommended default for generic text.Size chunks by tokens
Character count is a proxy for what actually matters: embedding models have a token limit and bill by the token. Sizing chunks by tokens keeps each chunk within the model’s limit and makes cost predictable. Use a token-based splitter when a chunk must fit a specific model, or when you want to control token cost directly.TokenTextSplitter splits directly on a token window. It does not use the recursive separator list that RecursiveCharacterTextSplitter uses for character-based splitting:
The
cl100k_base encoding is the tokenizer used by several OpenAI models. Other providers tokenize differently, so token counts are approximate across models. Choose the encoding that matches your embedding model when exact limits matter. For more token-splitting options, see Split by tokens.Split on document structure
When a document has structure, such as Markdown, HTML, or source code, split along that structure. Structural boundaries keep related content together, and they let you capture the structure itself as metadata. Split source code along its own boundaries (functions, classes, and blocks) withfromLanguage:
Chunk by meaning
Semantic chunking places boundaries where the topic shifts rather than at a fixed size. It compares the embeddings of adjacent sentences and starts a new chunk when their similarity drops, which produces more coherent chunks for prose that lacks clear structure. The tradeoff is cost: it embeds the document while splitting it. Semantic chunking is available only in Python through the unmaintainedlangchain-experimental package. In TypeScript, prefer recursive or structure-aware splitting.
Retrieve more context than you embed
Small chunks match precisely, but they can leave out the surrounding context a model needs to answer. The small-to-big pattern separates the two concerns: embed and index small chunks for accurate matching, then return the larger passage that contains the matched chunk to the model. To implement it, store a reference to the parent passage in each small chunk’s metadata. After retrieval, look up and return the parent passage instead of the matched chunk. TheParentDocumentRetriever helper implements this pattern and lives in the @langchain/classic package.
Attach metadata to chunks
Metadata on each chunk enables filtering and citation at retrieval time. Carry the source, the section, and the chunk’s position so you can scope searches and attribute answers. Metadata from each sourceDocument carries through to its chunks. Add custom fields to the source documents before splitting so every chunk inherits them:
Choose a strategy
Match the strategy to your document type, then tune size and overlap by measuring.Evaluate your chunking
No chunking strategy is correct in the abstract; the right one is the one that retrieves the right context for your queries. Vary chunk size, overlap, and strategy, re-index, then compare retrieval and answer quality on a fixed dataset. See Evaluate a RAG application to set up the dataset and evaluators.See also
- Retrieval: concepts and RAG architectures
- Improve retrieval quality: metadata filters, hybrid search, MMR, and reranking
- RAG with Deep Agents: loading, indexing, and querying documents
- Text splitter integrations: the full catalog of splitters
- Evaluate a RAG application: measure retrieval and answer quality
Connect these docs to your agent of choice via MCP for real-time answers.

