Quickstart

The LLM Gateway lets you call models across configured providers through one standard endpoint with one LangSmith API key. This quickstart uses the OpenAI Chat Completions format to call an Anthropic model.

Beta: The LLM Gateway is in beta.

Prerequisites

Before you start, confirm that:

Your Organization admin has enabled the LLM Gateway. For bring-your-own-key models, the admin must also add the provider API key to workspace secrets. To set this up, see Admin setup.
You have a workspace-scoped LangSmith API key attached to a role with gateway:invoke and workspaces:read permissions. Ask your organization admin if you are unsure.

You can call a Gateway Credits model without a provider secret. The example below uses a bring-your-own-key Anthropic model.

1. Set environment variables

Set the standard gateway base URL and your LangSmith API key:

export LANGSMITH_GATEWAY_BASE_URL="https://gateway.smith.langchain.com/v1"
export LANGSMITH_API_KEY="lsv2_..._....cbed3e"

The unified base URL accepts provider-prefixed bring-your-own-key model IDs, such as anthropic/claude-opus-5, and hosted model slugs, such as moonshotai/kimi-k3. The model ID determines the upstream route.

If your LangSmith account is on a regional instance, use the corresponding regional gateway.

To preserve a provider’s native API without format translation, use a direct provider route instead.

Using LangChain and Deep Agents

LangChain chat models and Deep Agents (including Deep Agents Code) support the gateway through two convenience environment variables:

export LANGSMITH_GATEWAY="true"
export LANGSMITH_GATEWAY_API_KEY="$LANGSMITH_API_KEY"

This routes all supported chat models through the gateway at https://gateway.smith.langchain.com. To use a different gateway (for example, the EU instance), set its URL instead of true:

export LANGSMITH_GATEWAY="https://eu.gateway.smith.langchain.com"
export LANGSMITH_GATEWAY_API_KEY="$LANGSMITH_API_KEY"

If the gateway is enabled but LANGSMITH_GATEWAY_API_KEY is unset, the gateway falls back to LANGSMITH_API_KEY.

You can also configure base URLs and API keys for individual providers. See the following accordion for provider support and interactions with provider-specific environment variables.

More details

Supported in Python only.
Supported chat models:
- Anthropic (langchain-anthropic >= 1.5.1)
- Baseten (langchain-baseten >= 0.2.3)
- Fireworks (langchain-fireworks >= 1.5.1)
- Google Gemini (langchain-google-genai >= 4.3.2)
- OpenAI (langchain-openai >= 1.4.1).
Provider-specific base URLs take precedence over the gateway, so you can still route an individual provider elsewhere. For example, with the gateway enabled, OPENAI_API_BASE sends OpenAI to that URL while every other provider continues to use the gateway:
export OPENAI_API_BASE="https://my.custom.gateway/openai/v2"

The following table shows how the base URL and key are resolved, using OpenAI as the example (other providers use their own *_API_BASE and *_API_KEY variables). GW default is https://gateway.smith.langchain.com/openai/v1.

`LANGSMITH_GATEWAY`	`LANGSMITH_GATEWAY_API_KEY`	`OPENAI_API_BASE`	`OPENAI_API_KEY`	`base_url=` kwarg	Resolved base URL	Resolved key
unset / `false`	—	—	—	—	`api.openai.com`	none
unset / `false`	✓	—	provider-key	—	`api.openai.com`	provider-key
`true`	✓	—	—	—	GW default	gateway-key
`true`	—	—	—	—	GW default	none
`true`	✓	—	provider-key	—	GW default	gateway-key
`true`	—	—	provider-key	—	GW default	provider-key
`true`	✓	`api.openai.com/v1`	provider-key	—	`api.openai.com/v1`	provider-key
`true`	✓	`api.openai.com/v1`	—	—	`api.openai.com/v1`	gateway-key
`true`	✓	`my.dev.gateway`	—	—	`my.dev.gateway`	gateway-key
`https://eu…`	✓	—	—	—	`eu…/openai/v1`	gateway-key
`https://eu…`	✓	—	—	`https://apac…`	`apac…`	gateway-key
`https://eu…`	✓	—	provider-key	`https://apac…`	`apac…`	provider-key

2. Make a call

curl "$LANGSMITH_GATEWAY_BASE_URL/chat/completions" \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-opus-5","messages":[{"role":"user","content":"ping"}]}'

import os

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LANGSMITH_GATEWAY_BASE_URL"],
    api_key=os.environ["LANGSMITH_API_KEY"],
)
response = client.chat.completions.create(
    model="anthropic/claude-opus-5",
    messages=[{"role": "user", "content": "ping"}],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.LANGSMITH_GATEWAY_BASE_URL,
  apiKey: process.env.LANGSMITH_API_KEY,
});
const response = await client.chat.completions.create({
  model: "anthropic/claude-opus-5",
  messages: [{ role: "user", content: "ping" }],
});
console.log(response.choices[0].message.content);

import os

from langchain.agents import create_agent
from langchain.chat_models import init_chat_model

model = init_chat_model(
    model="anthropic/claude-opus-5",
    model_provider="openai",
    base_url=os.environ["LANGSMITH_GATEWAY_BASE_URL"],
    api_key=os.environ["LANGSMITH_API_KEY"],
)
agent = create_agent(model=model, system_prompt="You are a helpful assistant.")
result = agent.invoke({"messages": [{"role": "user", "content": "ping"}]})
print(result["messages"][-1].content)

A 200 response with a chat completion confirms that the gateway, your API key, role permissions, and selected model route are working.

3. View your trace

Open the LangSmith UI and navigate to the tracing project named gateway or gateway-<short_api_key>-<api_key_id> in the workspace associated with your API key. You should see a new trace for the call you just made.

If your application also emits its own LangSmith traces, for example, through LangChain or LangGraph tracing, the gateway-side trace and your application trace appear as separate runs. Linking gateway traces to the parent application run is not yet supported.

4. Set a spend policy (optional)

Go to Settings → Gateway → LLM Gateway in LangSmith to create a spend policy. For example, you can set a daily $10 cap on your API key. When the cap is reached, the gateway returns a 402 response with the message: "Request blocked by gateway policies: R&D Spend Cap". See Spend policies for the full guide on policy dimensions, time windows, and conflict resolution.

How the gateway handles requests

The gateway performs these steps for each standard endpoint request:

Authenticates the request using the LangSmith API key.
Selects a hosted model or configured bring-your-own-key provider from the model ID.
Resolves the upstream credential. Hosted models use Gateway Credits, while bring-your-own-key models use workspace Provider Secrets.
Evaluates active policies, including spend limits, PII redaction, and secrets redaction.
Translates the request and response when the selected provider uses a different API format.
Traces the call to LangSmith, including token counts, cost, and policy events.

Next steps

Set up coding agents: route Claude Code, Codex, Gemini CLI, or Deep Agents Code through the gateway.
API formats: use Chat Completions, Messages, or Responses through the standard endpoint.
Direct model access: use provider-native request and response formats.
Spend policies: configure cost limits across your organization.
Data protection: prevent sensitive data from reaching providers.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Core capabilities

Administration and governance

Advanced

Prerequisites

1. Set environment variables

Using LangChain and Deep Agents

2. Make a call

3. View your trace

4. Set a spend policy (optional)

How the gateway handles requests

Next steps

​Prerequisites

​1. Set environment variables

​Using LangChain and Deep Agents

​2. Make a call

​3. View your trace

​4. Set a spend policy (optional)

​How the gateway handles requests

​Next steps

Prerequisites

1. Set environment variables

Using LangChain and Deep Agents

2. Make a call

3. View your trace

4. Set a spend policy (optional)

How the gateway handles requests

Next steps