LangChain v1.0Welcome to the new LangChain documentation! If you encounter any issues or have feedback, please open an issue so we can improve. Archived v0 documentation can be found here.See the release notes and migration guide for a complete list of changes and instructions on how to upgrade your code.


What can middleware do?
Monitor
Track agent behavior with logging, analytics, and debugging
Modify
Transform prompts, tool selection, and output formatting
Control
Add retries, fallbacks, and early termination logic
Enforce
Apply rate limits, guardrails, and PII detection
create_agent
:
Built-in middleware
LangChain provides prebuilt middleware for common use cases:Summarization
Automatically summarize conversation history when approaching token limits.Perfect for:
- Long-running conversations that exceed context windows
- Multi-turn dialogues with extensive history
- Applications where preserving full conversation context matters
Configuration options
Configuration options
Model for generating summaries
Token threshold for triggering summarization
Recent messages to preserve
Custom token counting function. Defaults to character-based counting.
Custom prompt template. Uses built-in template if not specified.
Prefix for summary messages
Human-in-the-loop
Pause agent execution for human approval, editing, or rejection of tool calls before they execute.Perfect for:
- High-stakes operations requiring human approval (database writes, financial transactions)
- Compliance workflows where human oversight is mandatory
- Long running conversations where human feedback is used to guide the agent
Configuration options
Configuration options
Mapping of tool names to approval configs. Values can be
True
(interrupt with default config), False
(auto-approve), or an InterruptOnConfig
object.Prefix for action request descriptions
InterruptOnConfig
options:List of allowed decisions:
"approve"
, "edit"
, or "reject"
Static string or callable function for custom description
Important: Human-in-the-loop middleware requires a checkpointer to maintain state across interruptions.See the human-in-the-loop documentation for complete examples and integration patterns.
Anthropic prompt caching
Reduce costs by caching repetitive prompt prefixes with Anthropic models.Perfect for:
- Applications with long, repeated system prompts
- Agents that reuse the same context across invocations
- Reducing API costs for high-volume deployments
Learn more about Anthropic Prompt Caching strategies and limitations.
Configuration options
Configuration options
Cache type. Only
"ephemeral"
is currently supported.Time to live for cached content. Valid values:
"5m"
or "1h"
Minimum number of messages before caching starts
Behavior when using non-Anthropic models. Options:
"ignore"
, "warn"
, or "raise"
Model call limit
Limit the number of model calls to prevent infinite loops or excessive costs.Perfect for:
- Preventing runaway agents from making too many API calls
- Enforcing cost controls on production deployments
- Testing agent behavior within specific call budgets
Configuration options
Configuration options
Maximum model calls across all runs in a thread. Defaults to no limit.
Maximum model calls per single invocation. Defaults to no limit.
Behavior when limit is reached. Options:
"end"
(graceful termination) or "error"
(raise exception)Tool call limit
Limit the number of tool calls to specific tools or all tools.Perfect for:
- Preventing excessive calls to expensive external APIs
- Limiting web searches or database queries
- Enforcing rate limits on specific tool usage
Configuration options
Configuration options
Specific tool to limit. If not provided, limits apply to all tools.
Maximum tool calls across all runs in a thread. Defaults to no limit.
Maximum tool calls per single invocation. Defaults to no limit.
Behavior when limit is reached. Options:
"end"
(graceful termination) or "error"
(raise exception)Model fallback
Automatically fallback to alternative models when the primary model fails.Perfect for:
- Building resilient agents that handle model outages
- Cost optimization by falling back to cheaper models
- Provider redundancy across OpenAI, Anthropic, etc.
Configuration options
Configuration options
PII detection
Detect and handle Personally Identifiable Information in conversations.Perfect for:
- Healthcare and financial applications with compliance requirements
- Customer service agents that need to sanitize logs
- Any application handling sensitive user data
Configuration options
Configuration options
Type of PII to detect. Can be a built-in type (
email
, credit_card
, ip
, mac_address
, url
) or a custom type name.How to handle detected PII. Options:
"block"
- Raise exception when detected"redact"
- Replace with[REDACTED_TYPE]
"mask"
- Partially mask (e.g.,****-****-****-1234
)"hash"
- Replace with deterministic hash
Custom detector function or regex pattern. If not provided, uses built-in detector for the PII type.
Check user messages before model call
Check AI messages after model call
Check tool result messages after execution
Planning
Add todo list management capabilities for complex multi-step tasks.This middleware automatically provides agents with a
write_todos
tool and system prompts to guide effective task planning.Configuration options
Configuration options
LLM tool selector
Use an LLM to intelligently select relevant tools before calling the main model.Perfect for:
- Agents with many tools (10+) where most aren’t relevant per query
- Reducing token usage by filtering irrelevant tools
- Improving model focus and accuracy
Configuration options
Configuration options
Model for tool selection. Can be a model string or
BaseChatModel
instance. Defaults to the agent’s main model.Instructions for the selection model. Uses built-in prompt if not specified.
Maximum number of tools to select. Defaults to no limit.
List of tool names to always include in the selection
Tool retry
Automatically retry failed tool calls with configurable exponential backoff.Perfect for:
- Handling transient failures in external API calls
- Improving reliability of network-dependent tools
- Building resilient agents that gracefully handle temporary errors
Configuration options
Configuration options
Maximum number of retry attempts after the initial call (3 total attempts with default)
Optional list of tools or tool names to apply retry logic to. If
None
, applies to all tools.Either a tuple of exception types to retry on, or a callable that takes an exception and returns
True
if it should be retried.Behavior when all retries are exhausted. Options:
"return_message"
- Return a ToolMessage with error details (allows LLM to handle failure)"raise"
- Re-raise the exception (stops agent execution)- Custom callable - Function that takes the exception and returns a string for the ToolMessage content
Multiplier for exponential backoff. Each retry waits
initial_delay * (backoff_factor ** retry_number)
seconds. Set to 0.0 for constant delay.Initial delay in seconds before first retry
Maximum delay in seconds between retries (caps exponential backoff growth)
Whether to add random jitter (±25%) to delay to avoid thundering herd
LLM tool emulator
Emulate tool execution using an LLM for testing purposes, replacing actual tool calls with AI-generated responses.Perfect for:
- Testing agent behavior without executing real tools
- Developing agents when external tools are unavailable or expensive
- Prototyping agent workflows before implementing actual tools
Configuration options
Configuration options
List of tool names (str) or BaseTool instances to emulate. If
None
(default), ALL tools will be emulated. If empty list, no tools will be emulated.Model to use for generating emulated tool responses. Can be a model identifier string or BaseChatModel instance.
Context editing
Manage conversation context by trimming, summarizing, or clearing tool uses.Perfect for:
- Long conversations that need periodic context cleanup
- Removing failed tool attempts from context
- Custom context management strategies
Configuration options
Configuration options
List of
ContextEdit
strategies to applyToken counting method. Options:
"approximate"
or "model"
ClearToolUsesEdit
options:Token count that triggers the edit
Minimum tokens to reclaim
Number of recent tool results to preserve
Whether to clear tool call parameters
List of tool names to exclude from clearing
Placeholder text for cleared outputs
Custom middleware
Build custom middleware by implementing hooks that run at specific points in the agent execution flow. You can create middleware in two ways:- Decorator-based - Quick and simple for single-hook middleware
- Class-based - More powerful for complex middleware with multiple hooks
Decorator-based middleware
For simple middleware that only needs a single hook, decorators provide the quickest way to add functionality:Available decorators
Node-style (run at specific execution points):@before_agent
- Before agent starts (once per invocation)@before_model
- Before each model call@after_model
- After each model response@after_agent
- After agent completes (once per invocation)
@wrap_model_call
- Around each model call@wrap_tool_call
- Around each tool call
@dynamic_prompt
- Generates dynamic system prompts (equivalent to@wrap_model_call
that modifies the prompt)
When to use decorators
Use decorators when
- You need a single hook
- No complex configuration
Use classes when
- Multiple hooks needed
- Complex configuration
- Reusable across projects (config on init)
Class-based middleware
Two hook styles
Node-style hooks
Run sequentially at specific execution points. Use for logging, validation, and state updates.
Wrap-style hooks
Intercept execution with full control over handler calls. Use for retries, caching, and transformation.
Node-style hooks
Run at specific points in the execution flow:before_agent
- Before agent starts (once per invocation)before_model
- Before each model callafter_model
- After each model responseafter_agent
- After agent completes (up to once per invocation)
Wrap-style hooks
Intercept execution and control when the handler is called:wrap_model_call
- Around each model callwrap_tool_call
- Around each tool call
Custom state schema
Middleware can extend the agent’s state with custom properties. Define a custom state type and set it as thestate_schema
:
Execution order
When using multiple middleware, understanding execution order is important:Execution flow (click to expand)
Execution flow (click to expand)
Before hooks run in order:
middleware1.before_agent()
middleware2.before_agent()
middleware3.before_agent()
middleware1.before_model()
middleware2.before_model()
middleware3.before_model()
middleware1.wrap_model_call()
→middleware2.wrap_model_call()
→middleware3.wrap_model_call()
→ model
middleware3.after_model()
middleware2.after_model()
middleware1.after_model()
middleware3.after_agent()
middleware2.after_agent()
middleware1.after_agent()
before_*
hooks: First to lastafter_*
hooks: Last to first (reverse)wrap_*
hooks: Nested (first middleware wraps all others)
Agent jumps
To exit early from middleware, return a dictionary withjump_to
:
"end"
: Jump to the end of the agent execution"tools"
: Jump to the tools node"model"
: Jump to the model node (or the firstbefore_model
hook)
before_model
or after_model
, jumping to "model"
will cause all before_model
middleware to run again.
To enable jumping, decorate your hook with @hook_config(can_jump_to=[...])
:
Best practices
- Keep middleware focused - each should do one thing well
- Handle errors gracefully - don’t let middleware errors crash the agent
- Use appropriate hook types:
- Node-style for sequential logic (logging, validation)
- Wrap-style for control flow (retry, fallback, caching)
- Clearly document any custom state properties
- Unit test middleware independently before integrating
- Consider execution order - place critical middleware first in the list
- Use built-in middleware when possible, don’t reinvent the wheel :)
Examples
Dynamically selecting tools
Select relevant tools at runtime to improve performance and accuracy.Benefits:
- Shorter prompts - Reduce complexity by exposing only relevant tools
- Better accuracy - Models choose correctly from fewer options
- Permission control - Dynamically filter tools based on user access
Additional resources
- Middleware API reference - Complete guide to custom middleware
- Human-in-the-loop - Add human review for sensitive operations
- Testing agents - Strategies for testing safety mechanisms