Skip to main content
Chunking splits your source documents into smaller pieces before you embed and index them. Chunk boundaries decide what a retriever can return: a chunk that mixes several topics dilutes its embedding, and a chunk that splits one idea across two pieces hides context from the model. Chunking is one of the largest factors in RAG quality, and one of the least obvious to tune. This guide covers how to choose and tune a chunking strategy. For the mechanics of each splitter, see the text splitter integrations. The examples assume docs is a list of loaded Document objects, as produced in RAG with Deep Agents.
Chunking choices are empirical. Trace and score them with LangSmith: the RAG evaluation tutorial shows how to measure retrieval and answer quality on a dataset, so you can compare strategies instead of guessing.

Balance chunk size and overlap

Chunk size trades precision against context, and there is no universal best value. It depends on your documents, your embedding model, and the questions you expect.
  • Small chunks produce focused embeddings and precise matches, but a single chunk may lack the surrounding context the model needs to answer.
  • Large chunks carry more context per chunk, but their embeddings blur together multiple topics, which makes them harder to match and spends more tokens per retrieved chunk.
Chunk overlap repeats a portion of text between adjacent chunks so that an idea split at a boundary still appears whole in at least one chunk. A common starting point is an overlap of 10% to 20% of the chunk size. Treat these as starting points and confirm them by evaluating your chunking.

Start with recursive character splitting

Start with RecursiveCharacterTextSplitter. It splits on a descending list of separators (paragraphs, then sentences, then words), which keeps semantically related text together whenever it fits within the target size. This is the recommended default for generic text.

Size chunks by tokens

Character count is a proxy for what actually matters: embedding models have a token limit and bill by the token. Sizing chunks by tokens keeps each chunk within the model’s limit and makes cost predictable. Use a token-based splitter when a chunk must fit a specific model, or when you want to control token cost directly. TokenTextSplitter splits directly on a token window. It does not use the recursive separator list that RecursiveCharacterTextSplitter uses for character-based splitting:
The cl100k_base encoding is the tokenizer used by several OpenAI models. Other providers tokenize differently, so token counts are approximate across models. Choose the encoding that matches your embedding model when exact limits matter. For more token-splitting options, see Split by tokens.

Split on document structure

When a document has structure, such as Markdown, HTML, or source code, split along that structure. Structural boundaries keep related content together, and they let you capture the structure itself as metadata. Split source code along its own boundaries (functions, classes, and blocks) with fromLanguage:
For Markdown and other structure-aware splitters, see the text splitter integrations.

Chunk by meaning

Semantic chunking places boundaries where the topic shifts rather than at a fixed size. It compares the embeddings of adjacent sentences and starts a new chunk when their similarity drops, which produces more coherent chunks for prose that lacks clear structure. The tradeoff is cost: it embeds the document while splitting it. Semantic chunking is available only in Python through the unmaintained langchain-experimental package. In TypeScript, prefer recursive or structure-aware splitting.

Retrieve more context than you embed

Small chunks match precisely, but they can leave out the surrounding context a model needs to answer. The small-to-big pattern separates the two concerns: embed and index small chunks for accurate matching, then return the larger passage that contains the matched chunk to the model. To implement it, store a reference to the parent passage in each small chunk’s metadata. After retrieval, look up and return the parent passage instead of the matched chunk. The ParentDocumentRetriever helper implements this pattern and lives in the @langchain/classic package.

Attach metadata to chunks

Metadata on each chunk enables filtering and citation at retrieval time. Carry the source, the section, and the chunk’s position so you can scope searches and attribute answers. Metadata from each source Document carries through to its chunks. Add custom fields to the source documents before splitting so every chunk inherits them:

Choose a strategy

Match the strategy to your document type, then tune size and overlap by measuring.

Evaluate your chunking

No chunking strategy is correct in the abstract; the right one is the one that retrieves the right context for your queries. Vary chunk size, overlap, and strategy, re-index, then compare retrieval and answer quality on a fixed dataset. See Evaluate a RAG application to set up the dataset and evaluators.

See also