Prompt caching
Reduce inference latency and input token costs by caching frequently reused prompt prefixes on Amazon Bedrock.BedrockPromptCachingMiddleware enables caching through model_settings. ChatBedrock and ChatBedrockConverse then translate that into the correct AWS wire format at request time. Cache checkpoints are placed after the system prompt, tool definitions, and the most recent message where supported, so that the model can skip recomputation of previously seen content on subsequent requests. Cache placement varies by API and model family: for example, Nova skips some tool definition and tool-result cases.
Prompt caching is useful for the following:
- Multi-turn conversations with long, consistent system prompts
- Agents with many tool definitions that remain constant across invocations
- Document-based Q&A where users ask multiple questions over the same uploaded context
- Batch processing workloads with repeated static content
- Anthropic Claude
- Amazon Nova
Learn more about AWS Bedrock prompt caching strategies and limitations. Cached content must exceed 1,024 tokens for a cache checkpoint to take effect, sometimes more depending on model. See supported models, regions, and limits.
BedrockPromptCachingMiddleware
ChatBedrockConverse
ChatBedrock
Configuration options
Configuration options
string
default:"ephemeral"
Cache type. For
ChatBedrock, only 'ephemeral' is currently supported. For ChatBedrockConverse, this value is ignored as the Converse API always uses "default" cache type.string
default:"5m"
Time to live for cached content. Valid values:
'5m' or '1h'. Note that Amazon Nova models only support '5m'.number
default:"0"
Minimum number of messages before caching starts.
string
default:"warn"
Behavior when using unsupported models. Options:
'ignore', 'warn', or 'raise'.Full example
Full example
The middleware caches content up to and including the latest message in each request. On subsequent requests within the TTL window (5 minutes or 1 hour), previously seen content is retrieved from cache rather than reprocessed, reducing costs and latency.How it works:
- First request: System prompt, tools, and the user message are sent to the API and cached
- Second request: The cached content is retrieved from cache. Only the new message needs to be processed
- This pattern continues for each turn, with each request reusing the cached conversation history
Prompt caching reduces API costs by caching tokens, but does not provide conversation memory. To persist conversation history across invocations, use a checkpointer like
MemorySaver.Model-specific behavior
The middleware handles differences between APIs and model families automatically:AgentCore Payments
AgentCore Payments is currently in preview and requires
bedrock-agentcore>=1.18.0.AgentCorePaymentsMiddleware detects the payment requirement, signs the payment via Amazon Bedrock AgentCore Payments, enforces session budget limits, and retries the request with payment credentials. This process is transparent to the agent.
AgentCore Payments middleware lives in the bedrock-agentcore package. The examples below install langchain-aws only to configure an Amazon Bedrock model; you can use the middleware with any model provider supported by LangChain agents.
AgentCore Payments middleware is useful for the following:
- Agents that access paid APIs without manual payment logic per tool
- Enforcing spending limits at the session level before any payment is signed
- Automatically recovering from payment errors (expired sessions, insufficient budget) via callbacks
- Supporting both SigV4 and bearer token (CUSTOM_JWT) authentication
- An AWS account with Amazon Bedrock AgentCore access
- An AWS Region where AgentCore Payments is available:
us-east-1,us-west-2,eu-central-1, orap-southeast-2. See Supported AWS Regions. - A configured PaymentManager resource (provides the ARN)
- A PaymentInstrument (wallet) provisioned for your user
- Python 3.10+
AgentCorePaymentsMiddleware
http_request tool can automatically retry requests to x402-compatible paid APIs after payment succeeds. When the tool receives a supported 402 response, the middleware handles payment signing, budget enforcement, and retry. If payment processing fails, the agent receives the configured or default payment error.
How it works
When a tool returns an HTTP 402 response with an x402 payload, the middleware:- Detects the payment requirement from the tool’s output
- Extracts the x402 payment details (amount, recipient, network)
- Validates the payment against the session budget (rejects if limit exceeded)
- Signs the payment via PaymentManager
- Injects the payment proof header into the tool’s arguments
- Waits briefly for on-chain propagation (configurable delay)
- Retries the original tool call with the payment header attached
Built-in tools
The middleware automatically registers these tools (available to the agent):
Set
provide_http_request=False if you bring your own HTTP tool.
Custom tool integration contract
For your own tools to work with auto-payment, they need two things: 1. Signal 402 (output): The tool must indicate a 402 response in its return value. Three formats are supported:PAYMENT_REQUIRED marker (recommended)
Raw JSON (fallback detection)
Custom handler
headers (input): The tool must have a headers parameter and forward it in its HTTP request. The middleware injects the payment header into tool_args["headers"] before retry:
Detection priority
When a tool returns, the middleware checks for 402 in this order:- Custom handler: If registered for the tool name via
custom_handlers, full control over detection PAYMENT_REQUIRED:marker: Explicit opt-in signal in content- Lenient fallback: Parses raw JSON for
statusCode: 402orx402Version+acceptsfields
MCP tool compatibility
MCP tools connected vialangchain-mcp-adapters can work with the middleware when the following conditions are met:
- The tool returns payment-related JSON (including
statusCode: 402) as text content inToolMessage.content(not inToolMessage.artifactorstructuredContent) - The tool accepts a
headersargument and forwards it in its outbound HTTP requests
ToolMessage.content, register a custom handler. MCP structuredContent stored in ToolMessage.artifact and MCP transport-level headers require adapter or transport integration outside this middleware.
Error handling
The middleware provides two layers of error control:Error handler callback (recommended)
The error handler callback is recommended because it keeps payment lifecycle complexity out of the agent’s reasoning. Without it, the agent receives error messages about expired sessions or missing instruments and must attempt to debug payment configuration, which wastes tokens and often fails. With the callback, your application code can resolve issues programmatically by creating sessions, refreshing instruments, or increasing budgets. When the callback returnsErrorResolution.RETRY, the middleware retries payment with the updated configuration. If the callback propagates the error, returns a custom message, raises, or exhausts its retries, the agent receives the configured or default payment error.
Callback flow:
Recommended resolution patterns:
Deterministic error messages (default)
When no callback is configured (or it returnsPROPAGATE), the agent receives a tailored error message with instructions not to retry:
All messages include
"Do not retry this call" and actionable guidance for the user.
Auto-session
Skip manual session creation. The middleware creates one lazily on the first 402:AgentCorePaymentsMiddleware per agent invocation (or per user request in a server). The middleware is not thread-safe.
Payment tool allowlist
Restrict which tools get payment processing:None (default), all tools are eligible.
Bearer token authentication
For payment managers usingCUSTOM_JWT authorizer:
Static token
Dynamic token provider (recommended)
user_id is optional (derived from JWT sub claim).
Disabling auto-payment
Use the middleware only for its built-in query tools without 402 interception:- Human-in-the-loop approval: Surface 402 details to the user and let them authorize each payment before it is signed
- Audit and compliance: Log payment requests for review without executing them, ensuring all transactions are explicitly approved
- Development and testing: Inspect raw 402 responses during integration without triggering real payments
- High-value transactions: Require manual review for payments above a certain threshold before proceeding
Custom handlers
Register customPaymentResponseHandler implementations for tools with non-standard output formats:
ToolMessage.content; parse it yourself. Do not pass built-in handlers (like GenericPaymentHandler) as custom handlers directly; they expect a different normalized shape.
Sync and async
The middleware provides both sync and async paths. LangGraph calls the right one automatically:
Install FastAPI to run the asynchronous web server example:
Async in FastAPI
Sync in a script
await asyncio.sleep()for the post-payment on-chain propagation delay (non-blocking)asyncio.to_thread()for PaymentManager signing calls (keeps event loop free)- Automatic
awaiton async error callbacks
Comparison: with vs without middleware
Without middleware (manual wrapping):- Write a wrapper function per tool type (~30-50 lines each)
- Handle 402 detection, x402 parsing, signing, retry manually
- Implement payment error handling for each wrapper
- Implement any required post-payment timing delay
- Create budget error messages for the agent
- Adding a new tool = another wrapper
Configuration reference
Learn more
- AgentCore Payments documentation
- Code samples: Agents that transact
- Blog: Introducing Amazon Bedrock AgentCore Payments
- Technical deep dive: AgentCore Payments and innovation in agentic commerce
Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

