Built-in middleware

LangChain provides prebuilt middleware for common use cases. Each middleware is production-ready and configurable for your specific needs.

Provider-agnostic middleware

The following middleware work with any LLM provider:

Middleware	Description
Summarization	Automatically summarize conversation history when approaching token limits.
Human-in-the-loop	Pause execution for human approval of tool calls.
Model call limit	Limit the number of model calls to prevent excessive costs.
Tool call limit	Control tool execution by limiting call counts.
Model fallback	Automatically fallback to alternative models when primary fails.
PII detection	Detect and handle Personally Identifiable Information (PII).
To-do list	Equip agents with task planning and tracking capabilities.
LLM tool selector	Use an LLM to select relevant tools before calling main model.
Tool retry	Automatically retry failed tool calls with exponential backoff.
LLM tool emulator	Emulate tool execution using anLLM for testing purposes.
Context editing	Manage conversation context by trimming or clearing tool uses.

Summarization

Automatically summarize conversation history when approaching token limits, preserving recent messages while compressing older context. Summarization is useful for the following:

Long-running conversations that exceed context windows.
Multi-turn dialogues with extensive history.
Applications where preserving full conversation context matters.

API reference: SummarizationMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[weather_tool, calculator_tool],
    middleware=[
        SummarizationMiddleware(
            model="gpt-4o-mini",
            trigger={"tokens": 4000},
            keep={"messages": 20},
        ),
    ],
)

Configuration options

model

string | BaseChatModel

required

Model for generating summaries. Can be a model identifier string (e.g., 'openai:gpt-4o-mini') or a BaseChatModel instance. See init_chat_model for more information.

trigger

dict | list[dict]

Conditions for triggering summarization. Can be:

A single condition dict (all properties must be met - AND logic)
A list of condition dicts (any condition must be met - OR logic)

Each condition can include:

fraction (float): Fraction of model’s context size (0-1)
tokens (int): Absolute token count
messages (int): Message count

At least one property must be specified per condition. If not provided, summarization will not trigger automatically.

keep

dict

default:"{messages: 20}"

How much context to preserve after summarization. Specify exactly one of:

fraction (float): Fraction of model’s context size to keep (0-1)
tokens (int): Absolute token count to keep
messages (int): Number of recent messages to keep

token_counter

function

Custom token counting function. Defaults to character-based counting.

summary_prompt

string

Custom prompt template for summarization. Uses built-in template if not specified. The template should include {messages} placeholder where conversation history will be inserted.

trim_tokens_to_summarize

number

default:"4000"

Maximum number of tokens to include when generating the summary. Messages will be trimmed to fit this limit before summarization.

summary_prefix

string

Prefix to add to the summary message. If not provided, a default prefix is used.

max_tokens_before_summary

number

deprecated

Deprecated: Use trigger: {"tokens": value} instead. Token threshold for triggering summarization.

messages_to_keep

number

deprecated

Deprecated: Use keep: {"messages": value} instead. Recent messages to preserve.

Full example

The summarization middleware monitors message token counts and automatically summarizes older messages when thresholds are reached.Trigger conditions control when summarization runs:

Single condition object (all properties must be met - AND logic)
Array of conditions (any condition must be met - OR logic)
Each condition can use fraction (of model’s context size), tokens (absolute count), or messages (message count)

Keep conditions control how much context to preserve (specify exactly one):

fraction - Fraction of model’s context size to keep
tokens - Absolute token count to keep
messages - Number of recent messages to keep

from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware


# Single condition: trigger if tokens >= 4000 AND messages >= 10
agent = create_agent(
    model="gpt-4o",
    tools=[weather_tool, calculator_tool],
    middleware=[
        SummarizationMiddleware(
            model="gpt-4o-mini",
            trigger={"tokens": 4000, "messages": 10},
            keep={"messages": 20},
        ),
    ],
)

# Multiple conditions
agent2 = create_agent(
    model="gpt-4o",
    tools=[weather_tool, calculator_tool],
    middleware=[
        SummarizationMiddleware(
            model="gpt-4o-mini",
            trigger=[
                {"tokens": 5000, "messages": 3},
                {"tokens": 3000, "messages": 6},
            ],
            keep={"messages": 20},
        ),
    ],
)

# Using fractional limits
agent3 = create_agent(
    model="gpt-4o",
    tools=[weather_tool, calculator_tool],
    middleware=[
        SummarizationMiddleware(
            model="gpt-4o-mini",
            trigger={"fraction": 0.8},
            keep={"fraction": 0.3},
        ),
    ],
)

Human-in-the-loop

Pause agent execution for human approval, editing, or rejection of tool calls before they execute. Human-in-the-loop is useful for the following:

High-stakes operations requiring human approval (e.g. database writes, financial transactions).
Compliance workflows where human oversight is mandatory.
Long-running conversations where human feedback guides the agent.

API reference: HumanInTheLoopMiddleware

Human-in-the-loop middleware requires a checkpointer to maintain state across interruptions.

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="gpt-4o",
    tools=[read_email_tool, send_email_tool],
    checkpointer=InMemorySaver(),
    middleware=[
        HumanInTheLoopMiddleware(
            interrupt_on={
                "send_email_tool": {
                    "allowed_decisions": ["approve", "edit", "reject"],
                },
                "read_email_tool": False,
            }
        ),
    ],
)

For complete examples, configuration options, and integration patterns, see the Human-in-the-loop documentation.

Model call limit

Limit the number of model calls to prevent infinite loops or excessive costs. Model call limit is useful for the following:

Preventing runaway agents from making too many API calls.
Enforcing cost controls on production deployments.
Testing agent behavior within specific call budgets.

API reference: ModelCallLimitMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import ModelCallLimitMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[...],
    middleware=[
        ModelCallLimitMiddleware(
            thread_limit=10,
            run_limit=5,
            exit_behavior="end",
        ),
    ],
)

Configuration options

thread_limit

number

Maximum model calls across all runs in a thread. Defaults to no limit.

run_limit

number

Maximum model calls per single invocation. Defaults to no limit.

exit_behavior

string

default:"end"

Behavior when limit is reached. Options: 'end' (graceful termination) or 'error' (raise exception)

Full example

The middleware tracks model calls across two scopes:

Thread limit - Max calls across all runs in a conversation thread (requires checkpointer)
Run limit - Max calls per single invocation (resets each turn)

Exit behaviors:

'end' - Graceful termination (default)
'error' - Raise/throw exception

from langchain.agents import create_agent
from langchain.agents.middleware import ModelCallLimitMiddleware
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, calculator_tool],
    checkpointer=InMemorySaver(),
    middleware=[
        ModelCallLimitMiddleware(
            thread_limit=10,
            run_limit=5,
            exit_behavior="end",
        ),
    ],
)

Tool call limit

Control agent execution by limiting the number of tool calls, either globally across all tools or for specific tools. Tool call limits are useful for the following:

Preventing excessive calls to expensive external APIs.
Limiting web searches or database queries.
Enforcing rate limits on specific tool usage.
Protecting against runaway agent loops.

API reference: ToolCallLimitMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import ToolCallLimitMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, database_tool],
    middleware=[
        # Global limit
        ToolCallLimitMiddleware(thread_limit=20, run_limit=10),
        # Tool-specific limit
        ToolCallLimitMiddleware(
            tool_name="search",
            thread_limit=5,
            run_limit=3,
        ),
    ],
)

Configuration options

tool_name

string

Name of specific tool to limit. If not provided, limits apply to all tools globally.

thread_limit

number

Maximum tool calls across all runs in a thread (conversation). Persists across multiple invocations with the same thread ID. Requires a checkpointer to maintain state. None means no thread limit.

run_limit

number

Maximum tool calls per single invocation (one user message → response cycle). Resets with each new user message. None means no run limit.Note: At least one of thread_limit or run_limit must be specified.

exit_behavior

string

default:"continue"

Behavior when limit is reached:

'continue' (default) - Block exceeded tool calls with error messages, let other tools and the model continue. The model decides when to end based on the error messages.
'error' - Raise a ToolCallLimitExceededError exception, stopping execution immediately
'end' - Stop execution immediately with a ToolMessage and AI message for the exceeded tool call. Only works when limiting a single tool; raises NotImplementedError if other tools have pending calls.

Full example

Specify limits with:

Thread limit - Max calls across all runs in a conversation (requires checkpointer)
Run limit - Max calls per single invocation (resets each turn)

Exit behaviors:

'continue' (default) - Block exceeded calls with error messages, agent continues
'error' - Raise exception immediately
'end' - Stop with ToolMessage + AI message (single-tool scenarios only)

from langchain.agents import create_agent
from langchain.agents.middleware import ToolCallLimitMiddleware


global_limiter = ToolCallLimitMiddleware(thread_limit=20, run_limit=10)
search_limiter = ToolCallLimitMiddleware(tool_name="search", thread_limit=5, run_limit=3)
database_limiter = ToolCallLimitMiddleware(tool_name="query_database", thread_limit=10)
strict_limiter = ToolCallLimitMiddleware(tool_name="scrape_webpage", run_limit=2, exit_behavior="error")

agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, database_tool, scraper_tool],
    middleware=[global_limiter, search_limiter, database_limiter, strict_limiter],
)

Model fallback

Automatically fallback to alternative models when the primary model fails. Model fallback is useful for the following:

Building resilient agents that handle model outages.
Cost optimization by falling back to cheaper models.
Provider redundancy across OpenAI, Anthropic, etc.

API reference: ModelFallbackMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import ModelFallbackMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[...],
    middleware=[
        ModelFallbackMiddleware(
            "gpt-4o-mini",
            "claude-3-5-sonnet-20241022",
        ),
    ],
)

Configuration options

first_model

string | BaseChatModel

required

First fallback model to try when the primary model fails. Can be a model identifier string (e.g., 'openai:gpt-4o-mini') or a BaseChatModel instance.

*additional_models

string | BaseChatModel

Additional fallback models to try in order if previous models fail

Full example

The middleware tries fallback models in order when the primary model fails.

from langchain.agents import create_agent
from langchain.agents.middleware import ModelFallbackMiddleware


agent = create_agent(
    model="gpt-4o",  # Primary model
    tools=[search_tool, calculator_tool],
    middleware=[
        ModelFallbackMiddleware(
            "gpt-4o-mini",
            "claude-3-5-sonnet-20241022",
            "claude-3-haiku-20240307",
        ),
    ],
)

PII detection

Detect and handle Personally Identifiable Information (PII) in conversations using configurable strategies. PII detection is useful for the following:

Healthcare and financial applications with compliance requirements.
Customer service agents that need to sanitize logs.
Any application handling sensitive user data.

API reference: PIIMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[...],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_input=True),
        PIIMiddleware("credit_card", strategy="mask", apply_to_input=True),
    ],
)

Configuration options

pii_type

string

required

Type of PII to detect. Can be a built-in type (email, credit_card, ip, mac_address, url) or a custom type name.

strategy

string

default:"redact"

How to handle detected PII. Options:

'block' - Raise exception when detected
'redact' - Replace with [REDACTED_TYPE]
'mask' - Partially mask (e.g., ****-****-****-1234)
'hash' - Replace with deterministic hash

detector

function | regex

Custom detector function or regex pattern. If not provided, uses built-in detector for the PII type.

apply_to_input

boolean

default:"True"

Check user messages before model call

apply_to_output

boolean

default:"False"

Check AI messages after model call

apply_to_tool_results

boolean

default:"False"

Check tool result messages after execution

Full example

The middleware supports detecting built-in PII types (email, credit_card, ip, mac_address, url) or custom types with regex patterns.Detection strategies:

'block' - Raise exception when detected
'redact' - Replace with [REDACTED_TYPE]
'mask' - Partially mask (e.g., ****-****-****-1234)
'hash' - Replace with deterministic hash

Application scope:

apply_to_input - Check user messages before model call
apply_to_output - Check AI messages after model call
apply_to_tool_results - Check tool result messages after execution

from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware


agent = create_agent(
    model="gpt-4o",
    tools=[database_tool, email_tool],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_input=True),
        PIIMiddleware("credit_card", strategy="mask", apply_to_input=True),
        PIIMiddleware("api_key", detector=r"sk-[a-zA-Z0-9]{32}", strategy="block"),
        PIIMiddleware("ssn", detector=r"\d{3}-\d{2}-\d{4}", strategy="hash", apply_to_tool_results=True),
    ],
)

To-do list

Equip agents with task planning and tracking capabilities for complex multi-step tasks. To-do lists are useful for the following:

Complex multi-step tasks requiring coordination across multiple tools.
Long-running operations where progress visibility is important.

This middleware automatically provides agents with a write_todos tool and system prompts to guide effective task planning.

API reference: TodoListMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import TodoListMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[read_file, write_file, run_tests],
    middleware=[TodoListMiddleware()],
)

Configuration options

system_prompt

string

Custom system prompt for guiding todo usage. Uses built-in prompt if not specified.

tool_description

string

Custom description for the write_todos tool. Uses built-in description if not specified.

Full example

Just as humans are more effective when they write down and track tasks, agents benefit from structured task management to break down complex problems.

from langchain.agents import create_agent
from langchain.agents.middleware import TodoListMiddleware
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool


@tool
def read_file(file_path: str) -> str:
    """Read contents of a file."""
    with open(file_path) as f:
        return f.read()


@tool
def write_file(file_path: str, content: str) -> str:
    """Write content to a file."""
    with open(file_path, 'w') as f:
        f.write(content)
    return f"Wrote {len(content)} characters to {file_path}"


agent = create_agent(
    model="gpt-4o",
    tools=[read_file, write_file],
    middleware=[TodoListMiddleware()],
)

result = agent.invoke({
    "messages": [HumanMessage("Refactor the authentication module")]
})

print(result["todos"])  # Track progress

LLM tool selector

Use an LLM to intelligently select relevant tools before calling the main model. LLM tool selectors are useful for the following:

Agents with many tools (10+) where most aren’t relevant per query.
Reducing token usage by filtering irrelevant tools.
Improving model focus and accuracy.

API reference: LLMToolSelectorMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import LLMToolSelectorMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[tool1, tool2, tool3, tool4, tool5, ...],
    middleware=[
        LLMToolSelectorMiddleware(
            model="gpt-4o-mini",
            max_tools=3,
            always_include=["search"],
        ),
    ],
)

Configuration options

model

string | BaseChatModel

Model for tool selection. Can be a model identifier string (e.g., 'openai:gpt-4o-mini') or a BaseChatModel instance. See init_chat_model for more information.Defaults to the agent’s main model.

system_prompt

string

Instructions for the selection model. Uses built-in prompt if not specified.

max_tools

number

Maximum number of tools to select. Defaults to no limit.

always_include

list[string]

List of tool names to always include in the selection

Full example

The middleware uses a (typically cheaper) LLM to analyze the user’s query and select the most relevant subset of tools.Benefits:

Shorter prompts - Reduce complexity by exposing only relevant tools
Better accuracy - Models choose correctly from fewer options
Cost savings - Use cheaper model for selection

from langchain.agents import create_agent
from langchain.agents.middleware import LLMToolSelectorMiddleware


agent = create_agent(
    model="gpt-4o",
    tools=[search_web, query_database, send_email, get_weather, ...],
    middleware=[
        LLMToolSelectorMiddleware(
            model="gpt-4o-mini",
            max_tools=3,
            always_include=["search_web"],
        ),
    ],
)

Tool retry

Automatically retry failed tool calls with configurable exponential backoff. Tool retry is useful for the following:

Handling transient failures in external API calls.
Improving reliability of network-dependent tools.
Building resilient agents that gracefully handle temporary errors.

API reference: ToolRetryMiddleware

from langchain.agents import create_agent
from langchain.agents.middleware import ToolRetryMiddleware

agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, database_tool],
    middleware=[
        ToolRetryMiddleware(
            max_retries=3,
            backoff_factor=2.0,
            initial_delay=1.0,
        ),
    ],
)

Configuration options

max_retries

number

default:"2"

Maximum number of retry attempts after the initial call (3 total attempts with default)

tools

list[BaseTool | str]

Optional list of tools or tool names to apply retry logic to. If None, applies to all tools.

retry_on

tuple[type[Exception], ...] | callable

default:"(Exception,)"

Either a tuple of exception types to retry on, or a callable that takes an exception and returns True if it should be retried.

on_failure

string | callable

default:"return_message"

Behavior when all retries are exhausted. Options:

'return_message' - Return a ToolMessage with error details (allows LLM to handle failure)
'raise' - Re-raise the exception (stops agent execution)
Custom callable - Function that takes the exception and returns a string for the ToolMessage content

backoff_factor

number

default:"2.0"

Multiplier for exponential backoff. Each retry waits initial_delay * (backoff_factor ** retry_number) seconds. Set to 0.0 for constant delay.

initial_delay

number

default:"1.0"

Initial delay in seconds before first retry

max_delay

number

default:"60.0"

Maximum delay in seconds between retries (caps exponential backoff growth)

jitter

boolean

default:"true"

Whether to add random jitter (±25%) to delay to avoid thundering herd

Full example

The middleware automatically retries failed tool calls with exponential backoff.Key configuration:

max_retries - Number of retry attempts (default: 2)
backoff_factor - Multiplier for exponential backoff (default: 2.0)
initial_delay - Starting delay in seconds (default: 1.0)
max_delay - Cap on delay growth (default: 60.0)
jitter - Add random variation (default: True)

Failure handling:

on_failure='return_message' - Return error message
on_failure='raise' - Re-raise exception
Custom callable - Function returning error message

from langchain.agents import create_agent
from langchain.agents.middleware import ToolRetryMiddleware


agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, database_tool, api_tool],
    middleware=[
        ToolRetryMiddleware(
            max_retries=3,
            backoff_factor=2.0,
            initial_delay=1.0,
            max_delay=60.0,
            jitter=True,
            tools=["api_tool"],
            retry_on=(ConnectionError, TimeoutError),
            on_failure="return_message",
        ),
    ],
)

LLM tool emulator

Emulate tool execution using an LLM for testing purposes, replacing actual tool calls with AI-generated responses. LLM tool emulators are useful for the following:

Testing agent behavior without executing real tools.
Developing agents when external tools are unavailable or expensive.
Prototyping agent workflows before implementing actual tools.

API reference: LLMToolEmulator

from langchain.agents import create_agent
from langchain.agents.middleware import LLMToolEmulator

agent = create_agent(
    model="gpt-4o",
    tools=[get_weather, search_database, send_email],
    middleware=[
        LLMToolEmulator(),  # Emulate all tools
    ],
)

Configuration options

tools

list[str | BaseTool]

List of tool names (str) or BaseTool instances to emulate. If None (default), ALL tools will be emulated. If empty list, no tools will be emulated.

model

string | BaseChatModel

default:"anthropic:claude-3-5-sonnet-latest"

Model to use for generating emulated tool responses. Can be a model identifier string (e.g., 'openai:gpt-4o-mini') or a BaseChatModel instance. See init_chat_model for more information.

Full example

The middleware uses an LLM to generate plausible responses for tool calls instead of executing the actual tools.

from langchain.agents import create_agent
from langchain.agents.middleware import LLMToolEmulator
from langchain_core.tools import tool


@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"Weather in {location}"

@tool
def send_email(to: str, subject: str, body: str) -> str:
    """Send an email."""
    return "Email sent"


# Emulate all tools
agent = create_agent(
    model="gpt-4o",
    tools=[get_weather, send_email],
    middleware=[LLMToolEmulator()],
)

# Emulate specific tools only
agent2 = create_agent(
    model="gpt-4o",
    tools=[get_weather, send_email],
    middleware=[LLMToolEmulator(tools=["get_weather"])],
)

# Use custom model
agent3 = create_agent(
    model="gpt-4o",
    tools=[get_weather, send_email],
    middleware=[LLMToolEmulator(model="claude-sonnet-4-5-20250929")],
)

Context editing

Manage conversation context by trimming, summarizing, or clearing tool uses. Context editing is useful for the following:

Long conversations that need periodic context cleanup.
Removing failed tool attempts from context.
Custom context management strategies.

API reference: ContextEditingMiddleware, ClearToolUsesEdit

from langchain.agents import create_agent
from langchain.agents.middleware import ContextEditingMiddleware, ClearToolUsesEdit

agent = create_agent(
    model="gpt-4o",
    tools=[...],
    middleware=[
        ContextEditingMiddleware(
            edits=[
                ClearToolUsesEdit(
                    trigger=100000,
                    keep=3,
                ),
            ],
        ),
    ],
)

Configuration options

edits

list[ContextEdit]

default:"[ClearToolUsesEdit()]"

List of ContextEdit strategies to apply

token_count_method

string

default:"approximate"

Token counting method. Options: 'approximate' or 'model'

ClearToolUsesEdit options:

trigger

number

default:"100000"

Token count that triggers the edit. When the conversation exceeds this token count, older tool outputs will be cleared.

clear_at_least

number

default:"0"

Minimum number of tokens to reclaim when the edit runs. If set to 0, clears as much as needed.

keep

number

default:"3"

Number of most recent tool results that must be preserved. These will never be cleared.

clear_tool_inputs

boolean

default:"False"

Whether to clear the originating tool call parameters on the AI message. When True, tool call arguments are replaced with empty objects.

exclude_tools

list[string]

default:"()"

List of tool names to exclude from clearing. These tools will never have their outputs cleared.

placeholder

string

default:"[cleared]"

Placeholder text inserted for cleared tool outputs. This replaces the original tool message content.

Full example

The middleware applies context editing strategies when token limits are reached. The most common strategy is ClearToolUsesEdit, which clears older tool results while preserving recent ones.How it works:

Monitor token count in conversation
When threshold is reached, clear older tool outputs
Keep most recent N tool results
Optionally preserve tool call arguments for context

from langchain.agents import create_agent
from langchain.agents.middleware import ContextEditingMiddleware, ClearToolUsesEdit


agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, calculator_tool, database_tool],
    middleware=[
        ContextEditingMiddleware(
            edits=[
                ClearToolUsesEdit(
                    trigger=2000,
                    keep=3,
                    clear_tool_inputs=False,
                    exclude_tools=[],
                    placeholder="[cleared]",
                ),
            ],
        ),
    ],
)

Provider-specific middleware

These middleware are optimized for specific LLM providers.

Anthropic

Middleware specifically designed for Anthropic’s Claude models.

Middleware	Description
Prompt caching	Reduce costs by caching repetitive prompt prefixes

Anthropic prompt caching

Reduce costs by caching repetitive prompt prefixes with Anthropic models. Prompt caching is useful for the following:

Applications with long, repeated system prompts.
Agents that reuse the same context across invocations.
Reducing API costs for high-volume deployments.

Learn more about Anthropic prompt caching strategies and limitations.

API reference: AnthropicPromptCachingMiddleware

from langchain_anthropic import ChatAnthropic
from langchain_anthropic.middleware import AnthropicPromptCachingMiddleware
from langchain.agents import create_agent

agent = create_agent(
    model=ChatAnthropic(model="claude-sonnet-4-5-20250929"),
    system_prompt="<Your long system prompt here>",
    middleware=[AnthropicPromptCachingMiddleware(ttl="5m")],
)

Configuration options

type

string

default:"ephemeral"

Cache type. Only 'ephemeral' is currently supported.

ttl

string

default:"5m"

Time to live for cached content. Valid values: '5m' or '1h'

min_messages_to_cache

number

default:"0"

Minimum number of messages before caching starts

unsupported_model_behavior

string

default:"warn"

Behavior when using non-Anthropic models. Options: 'ignore', 'warn', or 'raise'

Full example

from langchain_anthropic import ChatAnthropic
from langchain_anthropic.middleware import AnthropicPromptCachingMiddleware
from langchain.agents import create_agent
from langchain_core.messages import HumanMessage


LONG_PROMPT = """
Please be a helpful assistant.

<Lots more context ...>
"""

agent = create_agent(
    model=ChatAnthropic(model="claude-sonnet-4-5-20250929"),
    system_prompt=LONG_PROMPT,
    middleware=[AnthropicPromptCachingMiddleware(ttl="5m")],
)

# cache store
agent.invoke({"messages": [HumanMessage("Hi, my name is Bob")]})

# cache hit, system prompt is cached
agent.invoke({"messages": [HumanMessage("What's my name?")]})

OpenAI

Middleware specifically designed for OpenAI models.

Coming soon! Check back for OpenAI-specific middleware optimizations.

Edit the source of this page on GitHub.

Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.

LangChain v1.0

Get started

Core components

Middleware

Advanced usage

Use in production

Provider-agnostic middleware

Summarization

Human-in-the-loop

Model call limit

Tool call limit

Model fallback

PII detection

To-do list

LLM tool selector

Tool retry

LLM tool emulator

Context editing

Provider-specific middleware

Anthropic

Anthropic prompt caching

OpenAI

LangChain v1.0

Get started

Core components

Middleware

Advanced usage

Use in production

​Provider-agnostic middleware

​Summarization

​Human-in-the-loop

​Model call limit

​Tool call limit

​Model fallback

​PII detection

​To-do list

​LLM tool selector

​Tool retry

​LLM tool emulator

​Context editing

​Provider-specific middleware

​Anthropic

​Anthropic prompt caching

​OpenAI

Provider-agnostic middleware

Summarization

Human-in-the-loop

Model call limit

Tool call limit

Model fallback

PII detection

To-do list

LLM tool selector

Tool retry

LLM tool emulator

Context editing

Provider-specific middleware

Anthropic

Anthropic prompt caching

OpenAI