LLM Gateway

Use one LangSmith API key to call models across configured providers. Switch providers by changing the model ID, while the LLM Gateway traces every call and applies centralized governance policies.

Beta: The LLM Gateway is in beta.The gateway is also available on BYOC, where it runs inside your data plane. Send requests to your data plane endpoint behind the /gateway path prefix, and authenticate with an API key scoped to a workspace in that data plane. For more information, see Use a BYOC data plane.

Self-hosted availability: LLM Gateway is not included in the LangSmith v0.16.0 self-hosted stable release. It will be available in a future stable release. To express interest, submit the LLM Gateway self-hosted access request. You can also try LLM Gateway on v17 RC versions or BYOC (bring your own cloud) ahead of the stable release.

Make your first request

An administrator must enable the gateway, add a provider secret, and grant access once for your workspace. After setup, developers need only a workspace-scoped LangSmith API key.

Set your key and make a standard Chat Completions request. This example assumes the workspace has an Anthropic provider secret:

export LANGSMITH_API_KEY="lsv2_..._....cbed3e"

curl https://gateway.smith.langchain.com/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-sonnet-4-6","messages":[{"role":"user","content":"Hello!"}]}'

export LANGSMITH_API_KEY="lsv2_..._....cbed3e"

curl https://<data_plane_host>/gateway/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-sonnet-4-6","messages":[{"role":"user","content":"Hello!"}]}'

A 200 response confirms that the gateway, your LangSmith API key, permissions, and the selected provider secret are configured correctly. For Python, TypeScript, alternative API formats, and troubleshooting, follow the quickstart.

What the gateway provides

One key, multiple providers: Developers authenticate with a LangSmith API key instead of storing provider keys locally.
One request format, multiple models: Use Chat Completions, Messages, or Responses with models across configured providers.
Built-in observability: Every gateway call appears as a LangSmith trace.
Central governance: Apply spend limits, rate limits, and data-protection policies.

Use the standard API

Choose the request format already used by your application. The format does not limit which configured provider you can call.

API format	Endpoint
OpenAI Chat Completions	`POST /v1/chat/completions`
Anthropic Messages	`POST /v1/messages`
OpenAI Responses	`POST /v1/responses`

Set model to a provider-prefixed bring-your-own-key ID such as openai/gpt-5.4-mini or anthropic/claude-opus-5, or use a Gateway Credits model slug such as moonshotai/kimi-k3. The model ID determines the upstream route. When the selected provider uses a different native format, the gateway translates the request and response. On BYOC, the same paths sit behind the /gateway prefix, such as POST /gateway/v1/chat/completions. For base URLs, examples, translation behavior, regional endpoints, and BYOC data plane endpoints, see API formats.

Choose how credentials are managed

Option	Upstream credential	Setup and billing
Bring your own provider account	An administrator stores the provider key in workspace Provider Secrets.	The provider bills usage to your provider account.
Gateway Credits	LangChain owns the upstream credential.	No provider secret is required. Invocations are billed to your LangSmith account.

Go further

Quickstart

Make a request with cURL, Python, or TypeScript, then view its trace.

Administrator setup

Enable the gateway, add provider credentials, and grant developer access.

Need to use the gateway with prompts stored in the Prompt Hub? See Prompt Hub with the gateway for how to route Prompt Hub model calls through the gateway using two environment variables.Need provider-native request and response behavior? Use Direct model access to bypass the standardization layer. This is an advanced alternative to the standard API.For further questions, contact LangChain support.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Core capabilities

Administration and governance

Advanced

Make your first request

What the gateway provides

Use the standard API

Choose how credentials are managed

Go further

Quickstart

Administrator setup

​Make your first request

​What the gateway provides

​Use the standard API

​Choose how credentials are managed

​Go further

Quickstart

Administrator setup

Make your first request

What the gateway provides

Use the standard API

Choose how credentials are managed

Go further