> ## Documentation Index
> Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
> Use this file to discover all available pages before exploring further.

# NVIDIA middleware integration

> Integrate with the NVIDIA middleware using LangChain Python.

Middleware integrations and model-specific harness profiles for NVIDIA services and models. Use them to route calls across LangChain chat models or optimize Deep Agents behavior for NVIDIA Nemotron 3 Ultra. Learn more about [middleware](/oss/python/langchain/middleware/overview).

## Overview

| Integration                                                                            | Description                                                                       |
| -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| [Deep Agents profile for Nemotron 3 Ultra](#optimize-deep-agents-for-nemotron-3-ultra) | Apply model-specific prompts and middleware for more reliable agentic behavior    |
| [Model routing with NeMo Switchyard](#model-routing-with-nemo-switchyard)              | Route each model call to the target selected by a configured Switchyard algorithm |

## Optimize Deep Agents for Nemotron 3 Ultra

Deep Agents includes a built-in harness profile for [NVIDIA Nemotron 3 Ultra](https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b). The profile adds model-specific prompt guidance, tool descriptions, and middleware for tool calling, filesystem operations, retries, context management, and final answers. It was developed through [evaluation-driven harness tuning](https://developer.nvidia.com/blog/create-a-langchain-deep-agents-harness-profile-for-nvidia-nemotron-3-ultra-to-improve-performance/).

<Note>
  The built-in Nemotron 3 Ultra harness profile requires `deepagents>=0.7.0`.
</Note>

### What the profile changes

* Repairs common filesystem tool arguments and normalizes empty tool results.
* Adds continuation guidance when `read_file` returns a full page of results.
* Retries selected filesystem failures and model rate limits.
* Normalizes ChatNVIDIA messages, reasoning tags, and text-formatted tool calls.
* Adds progress, follow-up, entity-resolution, and final-answer safeguards.

### Setup

Install Deep Agents and the NVIDIA chat model integration:

```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
pip install -U "deepagents>=0.7.0" langchain-nvidia-ai-endpoints
```

Configure `NVIDIA_API_KEY` as described in the [ChatNVIDIA setup guide](/oss/python/integrations/chat/nvidia_ai_endpoints#access-the-nvidia-api-catalog).

### Use the built-in profile

Create a deep agent with the Nemotron 3 Ultra model. Deep Agents recognizes the model and applies the profile automatically, so you do not need to instantiate or pass its middleware:

```python Create a profiled deep agent icon="robot" theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from deepagents import create_deep_agent
from langchain_nvidia_ai_endpoints import ChatNVIDIA

model = ChatNVIDIA(model="nvidia/nemotron-3-ultra-550b-a55b")
agent = create_deep_agent(model=model)

result = agent.invoke({
    "messages": [
        {
            "role": "user",
            "content": "Inspect this project and summarize its architecture.",
        }
    ]
})
print(result["messages"][-1].content_blocks)
```

The profile is registered for Nemotron 3 Ultra model identifiers served through NVIDIA, Baseten, Fireworks, OpenRouter, Nebius, and Together. Registration is model-specific, so other models from these providers remain unchanged.

### API reference

* [Nemotron 3 Ultra harness profile source](https://github.com/langchain-ai/deepagents/blob/main/libs/deepagents/deepagents/profiles/harness/_nvidia_nemotron_3_ultra.py)
* [`HarnessProfile`](https://reference.langchain.com/python/deepagents/profiles/harness/harness_profiles#deepagents.profiles.harness.harness_profiles.HarnessProfile)

## Model routing with NeMo Switchyard

The experimental `SwitchyardRoutingMiddleware` routes each deep agent model call through a configured [NeMo Switchyard](https://github.com/NVIDIA-NeMo/Switchyard) `libsy` algorithm. Use it to combine LangChain chat models with different cost and performance profiles while Deep Agents continues to manage the agent loop, tools, state, and middleware composition.

<Warning>
  This integration is experimental. Its APIs and behavior are subject to breaking changes without notice. Review the [current limitations](https://github.com/langchain-ai/langchain-nvidia/tree/main/libs/switchyard#limitations) before using it in an application.
</Warning>

### Features

* Adapt any [`BaseChatModel`](https://reference.langchain.com/python/langchain-core/language_models/chat_models/BaseChatModel) as a Switchyard target with `LangChainLlmClient`.
* Use any algorithm exposed by the installed `nemo-switchyard` Python bindings.
* Preserve Deep Agents tool binding, callbacks, LangChain tracing, and structured output.
* Inspect the selected model and complete routing trace on each returned [`AIMessage`](https://reference.langchain.com/python/langchain-core/messages/ai/AIMessage).
* Route synchronous and asynchronous agent calls through the same algorithm.

### Setup

The integration requires Python 3.12 or newer, a source checkout of NeMo Switchyard, and a checkout of the `langchain-nvidia` repository. Configure credentials for every chat model that the router can select. The middleware itself does not require a provider API key.

#### Installation

Build Switchyard's `libsy` Python bindings from source, then install the integration with its Deep Agents and OpenRouter dependencies:

```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
git clone https://github.com/NVIDIA-NeMo/Switchyard.git
python -m pip install -e ./Switchyard

git clone https://github.com/langchain-ai/langchain-nvidia.git
python -m pip install -e "./langchain-nvidia/libs/switchyard[openrouter]"
```

The OpenRouter example requires `OPENROUTER_API_KEY`. Confirm that your account can access both configured models before running it.

<Note>
  `SwitchyardRoutingMiddleware` requires `deepagents>=0.7.4`.
</Note>

### Instantiation

Create two LangChain chat models, adapt them as Switchyard targets, and construct the routing middleware. The order of the targets passed to `stage_router` matters: pass the capable target first and the efficient target second.

```python Initialize middleware icon="arrows-shuffle" theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_openrouter import ChatOpenRouter

from switchyard.libsy import LlmTarget, algorithms
from langchain_nvidia_switchyard import LangChainLlmClient, SwitchyardRoutingMiddleware

efficient_model = ChatOpenRouter(model="nvidia/nemotron-3-ultra-550b-a55b")
capable_model = ChatOpenRouter(model="anthropic/claude-sonnet-4.6")

router = algorithms.stage_router(
    LlmTarget("capable", LangChainLlmClient(capable_model)),
    LlmTarget("efficient", LangChainLlmClient(efficient_model)),
    picker="efficient_first",
    confidence_threshold=0.5,
    recent_window=3,
)

middleware = SwitchyardRoutingMiddleware(router)
```

Stage routing is signal-driven. It can route ordinary turns to the efficient target and escalate turns with critical failed-tool signals to the capable target without making a separate judge-model call.

### Use with a deep agent

Pass the middleware to [`create_deep_agent`](https://reference.langchain.com/python/deepagents/graph/create_deep_agent). Deep Agents requires a base model, but the middleware replaces it for each routed call. Reuse one configured target to avoid constructing an unused model.

```python Agent with middleware icon="robot" theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from deepagents import create_deep_agent

agent = create_deep_agent(
    model=efficient_model,
    middleware=[middleware],
)

result = await agent.ainvoke({
    "messages": [
        {
            "role": "user",
            "content": "Summarize the important files in this project.",
        }
    ]
})
```

The asynchronous `ainvoke` path is canonical. Use `agent.invoke(...)` in ordinary synchronous applications, but use `await agent.ainvoke(...)` in notebooks, async web handlers, and async tests.

### Choose a routing algorithm

`SwitchyardRoutingMiddleware` accepts an opaque `switchyard.libsy.Algorithm`, so the middleware does not depend on a concrete routing strategy. Choose from the algorithms exposed by the installed bindings:

* **Stage routing**: Route from message-history signals, including assistant tool calls and tool results, without a judge by default.
* **LLM task classifier**: Call a judge model before selecting the efficient or capable target.
* **Random routing**: Select among weighted targets, with an optional seed for a reproducible process-local selection sequence.
* **No-op**: Check the middleware boundary without a provider call.

### Inspect routing decisions

Every routed `AIMessage` contains the complete ordered decision trace in `response_metadata["switchyard"]`:

```python Inspect routing icon="route" theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain.messages import AIMessage

message = next(
    message
    for message in reversed(result["messages"])
    if isinstance(message, AIMessage)
)

routing = message.response_metadata["switchyard"]
print(routing["selected_model"])
print(routing["decisions"])
```

Some algorithms make more than one decision. Use `decisions` for the full trace; `selected_model` contains only the final selection.

### API reference

* [`SwitchyardRoutingMiddleware(algorithm)`](https://github.com/langchain-ai/langchain-nvidia/tree/main/libs/switchyard#switchyardroutingmiddlewarealgorithm)
* [`LangChainLlmClient(model)`](https://github.com/langchain-ai/langchain-nvidia/tree/main/libs/switchyard#langchainllmclientmodel)

## See also

* [NVIDIA integrations](/oss/python/integrations/providers/nvidia)
* [`langchain-nvidia-switchyard` package README](https://github.com/langchain-ai/langchain-nvidia/tree/main/libs/switchyard#readme)
* [Create a Deep Agents harness profile for NVIDIA Nemotron 3 Ultra](https://developer.nvidia.com/blog/create-a-langchain-deep-agents-harness-profile-for-nvidia-nemotron-3-ultra-to-improve-performance/)
* [Deep Agents models and dynamic selection](/oss/python/deepagents/models)

***

<div className="source-links">
  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>

  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/middleware/nvidia.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>
</div>