Skip to main content
Middleware integrations for AWS services. Prompt caching is designed for models hosted on Amazon Bedrock, while AgentCore Payments works with LangGraph agents regardless of model provider. Learn more about middleware.

Prompt caching

Reduce inference latency and input token costs by caching frequently reused prompt prefixes on Amazon Bedrock. BedrockPromptCachingMiddleware enables caching through model_settings. ChatBedrock and ChatBedrockConverse then translate that into the correct AWS wire format at request time. Cache checkpoints are placed after the system prompt, tool definitions, and the most recent message where supported, so that the model can skip recomputation of previously seen content on subsequent requests. Cache placement varies by API and model family: for example, Nova skips some tool definition and tool-result cases. Prompt caching is useful for the following:
  • Multi-turn conversations with long, consistent system prompts
  • Agents with many tool definitions that remain constant across invocations
  • Document-based Q&A where users ask multiple questions over the same uploaded context
  • Batch processing workloads with repeated static content
Supported models:
  • Anthropic Claude
  • Amazon Nova
Learn more about AWS Bedrock prompt caching strategies and limitations. Cached content must exceed 1,024 tokens for a cache checkpoint to take effect, sometimes more depending on model. See supported models, regions, and limits.
API reference: BedrockPromptCachingMiddleware
ChatBedrockConverse
ChatBedrock
string
default:"ephemeral"
Cache type. For ChatBedrock, only 'ephemeral' is currently supported. For ChatBedrockConverse, this value is ignored as the Converse API always uses "default" cache type.
string
default:"5m"
Time to live for cached content. Valid values: '5m' or '1h'. Note that Amazon Nova models only support '5m'.
number
default:"0"
Minimum number of messages before caching starts.
string
default:"warn"
Behavior when using unsupported models. Options: 'ignore', 'warn', or 'raise'.
The middleware caches content up to and including the latest message in each request. On subsequent requests within the TTL window (5 minutes or 1 hour), previously seen content is retrieved from cache rather than reprocessed, reducing costs and latency.How it works:
  1. First request: System prompt, tools, and the user message are sent to the API and cached
  2. Second request: The cached content is retrieved from cache. Only the new message needs to be processed
  3. This pattern continues for each turn, with each request reusing the cached conversation history
Prompt caching reduces API costs by caching tokens, but does not provide conversation memory. To persist conversation history across invocations, use a checkpointer like MemorySaver.

Model-specific behavior

The middleware handles differences between APIs and model families automatically:

AgentCore Payments

AgentCore Payments is currently in preview and requires bedrock-agentcore>=1.18.0.
Autonomously handle x402 Payment Required responses in LangGraph agents. When a tool hits a paid API that returns HTTP 402, AgentCorePaymentsMiddleware detects the payment requirement, signs the payment via Amazon Bedrock AgentCore Payments, enforces session budget limits, and retries the request with payment credentials. This process is transparent to the agent. AgentCore Payments middleware lives in the bedrock-agentcore package. The examples below install langchain-aws only to configure an Amazon Bedrock model; you can use the middleware with any model provider supported by LangChain agents. AgentCore Payments middleware is useful for the following:
  • Agents that access paid APIs without manual payment logic per tool
  • Enforcing spending limits at the session level before any payment is signed
  • Automatically recovering from payment errors (expired sessions, insufficient budget) via callbacks
  • Supporting both SigV4 and bearer token (CUSTOM_JWT) authentication
For a guided setup of PaymentManager and instruments, see the AgentCore Payments getting started skill. Prerequisites:
  • An AWS account with Amazon Bedrock AgentCore access
  • An AWS Region where AgentCore Payments is available: us-east-1, us-west-2, eu-central-1, or ap-southeast-2. See Supported AWS Regions.
  • A configured PaymentManager resource (provides the ARN)
  • A PaymentInstrument (wallet) provisioned for your user
  • Python 3.10+
Installation:
API reference: AgentCorePaymentsMiddleware
With this setup, the built-in http_request tool can automatically retry requests to x402-compatible paid APIs after payment succeeds. When the tool receives a supported 402 response, the middleware handles payment signing, budget enforcement, and retry. If payment processing fails, the agent receives the configured or default payment error.

How it works

When a tool returns an HTTP 402 response with an x402 payload, the middleware:
  1. Detects the payment requirement from the tool’s output
  2. Extracts the x402 payment details (amount, recipient, network)
  3. Validates the payment against the session budget (rejects if limit exceeded)
  4. Signs the payment via PaymentManager
  5. Injects the payment proof header into the tool’s arguments
  6. Waits briefly for on-chain propagation (configurable delay)
  7. Retries the original tool call with the payment header attached

Built-in tools

The middleware automatically registers these tools (available to the agent): Set provide_http_request=False if you bring your own HTTP tool.

Custom tool integration contract

For your own tools to work with auto-payment, they need two things: 1. Signal 402 (output): The tool must indicate a 402 response in its return value. Three formats are supported:
PAYMENT_REQUIRED marker (recommended)
Raw JSON (fallback detection)
Custom handler
2. Accept and forward headers (input): The tool must have a headers parameter and forward it in its HTTP request. The middleware injects the payment header into tool_args["headers"] before retry:
Without this, the payment header is injected but never sent to the server.

Detection priority

When a tool returns, the middleware checks for 402 in this order:
  1. Custom handler: If registered for the tool name via custom_handlers, full control over detection
  2. PAYMENT_REQUIRED: marker: Explicit opt-in signal in content
  3. Lenient fallback: Parses raw JSON for statusCode: 402 or x402Version + accepts fields

MCP tool compatibility

MCP tools connected via langchain-mcp-adapters can work with the middleware when the following conditions are met:
  1. The tool returns payment-related JSON (including statusCode: 402) as text content in ToolMessage.content (not in ToolMessage.artifact or structuredContent)
  2. The tool accepts a headers argument and forwards it in its outbound HTTP requests
When these conditions are satisfied, the lenient fallback detection handles 402 responses automatically. For non-standard formats that are still exposed through ToolMessage.content, register a custom handler. MCP structuredContent stored in ToolMessage.artifact and MCP transport-level headers require adapter or transport integration outside this middleware.

Error handling

The middleware provides two layers of error control: The error handler callback is recommended because it keeps payment lifecycle complexity out of the agent’s reasoning. Without it, the agent receives error messages about expired sessions or missing instruments and must attempt to debug payment configuration, which wastes tokens and often fails. With the callback, your application code can resolve issues programmatically by creating sessions, refreshing instruments, or increasing budgets. When the callback returns ErrorResolution.RETRY, the middleware retries payment with the updated configuration. If the callback propagates the error, returns a custom message, raises, or exhausts its retries, the agent receives the configured or default payment error.
The callback can return: Callback flow:
PaymentErrorContext fields: Recommended resolution patterns:

Deterministic error messages (default)

When no callback is configured (or it returns PROPAGATE), the agent receives a tailored error message with instructions not to retry: All messages include "Do not retry this call" and actionable guidance for the user.

Auto-session

Skip manual session creation. The middleware creates one lazily on the first 402:
The session is created once and reused for all subsequent payments in that middleware instance. Create one AgentCorePaymentsMiddleware per agent invocation (or per user request in a server). The middleware is not thread-safe.

Payment tool allowlist

Restrict which tools get payment processing:
Tools not in the list pass through untouched. When None (default), all tools are eligible.

Bearer token authentication

For payment managers using CUSTOM_JWT authorizer:
Static token
Dynamic token provider (recommended)
With bearer auth, user_id is optional (derived from JWT sub claim).

Disabling auto-payment

Use the middleware only for its built-in query tools without 402 interception:
Common reasons to disable auto-payment:
  • Human-in-the-loop approval: Surface 402 details to the user and let them authorize each payment before it is signed
  • Audit and compliance: Log payment requests for review without executing them, ensuring all transactions are explicitly approved
  • Development and testing: Inspect raw 402 responses during integration without triggering real payments
  • High-value transactions: Require manual review for payments above a certain threshold before proceeding

Custom handlers

Register custom PaymentResponseHandler implementations for tools with non-standard output formats:
Custom handlers receive the raw ToolMessage.content; parse it yourself. Do not pass built-in handlers (like GenericPaymentHandler) as custom handlers directly; they expect a different normalized shape.

Sync and async

The middleware provides both sync and async paths. LangGraph calls the right one automatically: Install FastAPI to run the asynchronous web server example:
Async in FastAPI
Sync in a script
The async path uses:
  • await asyncio.sleep() for the post-payment on-chain propagation delay (non-blocking)
  • asyncio.to_thread() for PaymentManager signing calls (keeps event loop free)
  • Automatic await on async error callbacks

Comparison: with vs without middleware

Without middleware (manual wrapping):
  • Write a wrapper function per tool type (~30-50 lines each)
  • Handle 402 detection, x402 parsing, signing, retry manually
  • Implement payment error handling for each wrapper
  • Implement any required post-payment timing delay
  • Create budget error messages for the agent
  • Adding a new tool = another wrapper
With middleware:
Compatible tools that meet the custom tool integration contract are handled automatically.

Configuration reference

Learn more