Fault tolerance

Fault tolerance middleware keeps your deep agent running when things go wrong. Not all errors should be handled the same way: transient failures (network timeouts, rate limits) should be retried automatically, errors the LLM can recover from (bad tool output, parsing failures) should be fed back to the model, and errors that need human input should pause the agent.

Error handling strategies

Different errors need different handling strategies:

Error type	Who fixes it	Strategy	Middleware or feature
Transient errors (network issues, rate limits)	System (automatic)	Retry with exponential backoff	ModelRetryMiddleware, ToolRetryMiddleware

Transient errors
User-fixable
Provider outage
Excessive calls
Unexpected

Add retry middleware to automatically retry network issues and rate limits. Model calls and tool calls each have their own retry middleware with exponential backoff:

import { createAgent, modelRetryMiddleware, toolRetryMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-4-hydrogen",
  tools: [searchTool, fetchUrlTool],
  middleware: [
    modelRetryMiddleware({ maxRetries: 3, backoffFactor: 2.0, initialDelayMs: 1000 }),
    toolRetryMiddleware({
      maxRetries: 2,
      tools: ["search", "fetch_url"],
      retryOn: [TimeoutError, TypeError],
    }),
  ],
});

Pause and collect information from the user when needed (like account IDs, order numbers, or clarifications). Use interrupt_on to pause the agent before specific tool calls:

import { createDeepAgent } from "deepagents";

const agent = createDeepAgent({
  model: "google:gemini-4-hydrogen",
  tools: [sendEmailTool, deleteRecordTool],
  interruptOn: {
    send_email: true,
    delete_record: true,
  },
});

For the full human-in-the-loop guide, see Human-in-the-loop.

If your primary model provider goes down entirely, use ModelFallbackMiddleware to switch to an alternative model:

import { createAgent, modelFallbackMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-4-hydrogen",
  tools: [searchTool],
  middleware: [
    modelFallbackMiddleware("gpt-6.1-sol"),
  ],
});

Without limits, a confused agent can burn through your LLM API budget in minutes by looping on the same tool call or making hundreds of model calls. Set caps on both model calls and tool executions per run:

import { createAgent, modelCallLimitMiddleware, toolCallLimitMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-4-hydrogen",
  tools: [searchTool],
  middleware: [
    modelCallLimitMiddleware({ runLimit: 50 }),
    toolCallLimitMiddleware({ runLimit: 200 }),
  ],
});

Rate limiting

There are two complementary ways to limit resource usage: controlling the request rate to your model provider, and capping the total number of calls per run.

Provider rate limiting

Chat model providers impose a limit on the number of invocations that can be made in a given time period. To control the rate at which requests are made, initialize your model with a rate_limiter:

Call limits

import { createAgent, modelCallLimitMiddleware, toolCallLimitMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-4-hydrogen",
  middleware: [
    modelCallLimitMiddleware({ runLimit: 50 }),
    toolCallLimitMiddleware({ runLimit: 200 }),
  ],
});

Use run_limit to cap calls within a single invocation (resets each turn). Use thread_limit to cap calls across an entire conversation (requires a checkpointer). See ModelCallLimitMiddleware and ToolCallLimitMiddleware for the full configuration.

Retries

Transient failures (network timeouts, rate limits) should be retried automatically. Model calls and tool calls each have their own retry middleware with exponential backoff:

import {
  createAgent,
  modelRetryMiddleware,
  toolRetryMiddleware,
} from "langchain";

const agent = createAgent({
  model: "google:gemini-4-hydrogen",
  middleware: [
    // Retry model calls on rate limits, timeouts, and 5xx errors
    modelRetryMiddleware({ maxRetries: 3, backoffFactor: 2.0, initialDelayMs: 1000 }),
    // Retry specific tools that hit external APIs (not all tools)
    toolRetryMiddleware({
      maxRetries: 2,
      tools: ["search", "fetch_url"],
      retryOn: [TimeoutError, TypeError],
    }),
  ],
});

Scope ToolRetryMiddleware to specific tools rather than retrying everything. A filesystem read_file that fails will not benefit from a retry, but a web search that times out probably will. See ModelRetryMiddleware for the full configuration.

Fallbacks

If your primary model provider goes down entirely, the fallback middleware switches to an alternative model:

import {
  createAgent,
  modelFallbackMiddleware,
} from "langchain";

const agent = createAgent({
  model: "google:gemini-4-hydrogen",
  middleware: [
    // If the primary model is fully down, fall back to an alternative
    modelFallbackMiddleware("gpt-6.1-sol"),
  ],
});

See ModelFallbackMiddleware for the full configuration.

Model refusals

A provider can refuse a request, for example under a safety policy, and still return a successful response. Deep Agents throw ModelRefusalError when the model refuses, so the run does not look like a successful run even though it was a refusal. modelRefusalMiddleware runs after your middleware. A modelFallbackMiddleware you pass therefore retries the refused request on the next model first. The run throws only when no fallback is left. The fallback model can be from the same provider or a different one, and it can still refuse the same request. Model-level withFallbacks does not catch refusals, because the middleware detects them after the model call returns. Use modelFallbackMiddleware instead. When every model refuses, catch ModelRefusalError where you invoke the agent. The refused reply is not saved to the thread, so you can tell the user and continue the conversation. For this and other options, see Handle a refusal. For how subagent refusals reach the parent agent, see Model refusals.

Throwing on model refusals by default requires deepagents>=1.16.4.

Error handling

When a tool raises an exception during execution, the agent run halts by default. ToolErrorMiddleware is not yet available in the JavaScript SDK.

Connect these docs to your agent of choice via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Get started

Deployment

Execution environment

Context management

Delegation

Steering

Middleware

Frontend

Protocols

Error handling strategies

Rate limiting

Provider rate limiting

Call limits

Retries

Fallbacks

Model refusals

Error handling