API formats

The standard LLM Gateway API supports three request and response formats. Choose the format your application already uses, then call models across configured providers with provider-prefixed model IDs.

Beta: The LLM Gateway is in beta.

Compare API formats

API format	Base URL	Prompt endpoint	Compatible client
OpenAI Chat Completions	`https://gateway.smith.langchain.com/v1`	`POST /chat/completions`	OpenAI-compatible Chat Completions clients
Anthropic Messages	`https://gateway.smith.langchain.com`	`POST /v1/messages`	Anthropic Messages clients
OpenAI Responses	`https://gateway.smith.langchain.com/v1`	`POST /responses`	OpenAI-compatible Responses clients

All formats authenticate with a workspace-scoped LangSmith API key. Pass it as the provider API key or as an Authorization: Bearer token. Set model to <provider>/<model>, such as openai/gpt-5.4-mini or anthropic/claude-sonnet-4-6. The request format does not limit which configured provider you can call.

Use Chat Completions

Point an OpenAI-compatible client at https://gateway.smith.langchain.com/v1. For the full request and response schema, see the OpenAI Chat Completions API.

curl https://gateway.smith.langchain.com/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-sonnet-4-6","messages":[{"role":"user","content":"Hello!"}]}'

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.smith.langchain.com/v1",
    api_key=os.environ["LANGSMITH_API_KEY"],
)
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Hello!"}],
)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.smith.langchain.com/v1",
  apiKey: process.env.LANGSMITH_API_KEY,
});
const response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-4-6",
  messages: [{ role: "user", content: "Hello!" }],
});

Use Messages

Point an Anthropic client at https://gateway.smith.langchain.com. For the full request and response schema, see the Anthropic Messages API.

curl https://gateway.smith.langchain.com/v1/messages \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"openai/gpt-5.4-mini","max_tokens":1024,"messages":[{"role":"user","content":"Hello!"}]}'

import os

import anthropic

client = anthropic.Anthropic(
    base_url="https://gateway.smith.langchain.com",
    api_key=os.environ["LANGSMITH_API_KEY"],
)
message = client.messages.create(
    model="openai/gpt-5.4-mini",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "https://gateway.smith.langchain.com",
  apiKey: process.env.LANGSMITH_API_KEY,
});
const message = await client.messages.create({
  model: "openai/gpt-5.4-mini",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello!" }],
});

Use Responses

Point an OpenAI-compatible client at https://gateway.smith.langchain.com/v1. For the full request and response schema, see the OpenAI Responses API.

curl https://gateway.smith.langchain.com/v1/responses \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-sonnet-4-6","input":"Hello!"}'

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.smith.langchain.com/v1",
    api_key=os.environ["LANGSMITH_API_KEY"],
)
response = client.responses.create(
    model="anthropic/claude-sonnet-4-6",
    input="Hello!",
)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.smith.langchain.com/v1",
  apiKey: process.env.LANGSMITH_API_KEY,
});
const response = await client.responses.create({
  model: "anthropic/claude-sonnet-4-6",
  input: "Hello!",
});

Understand translation behavior

The endpoint determines the format your application sends and receives. The model ID determines the upstream provider.

When the provider supports the selected format natively, the gateway preserves that format.
Otherwise, the gateway translates the request into a format supported by the provider and translates the response back, including streaming responses.
Translation can reject fields that cannot be represented in the target provider format. Use Direct model access when provider-native behavior is required.

Every request resolves the same Provider Secrets, policies, and tracing configuration regardless of format.

List models

Call GET /v1/models to list models available across providers configured for the workspace. The gateway aggregates each provider’s model catalog into a single OpenAI-compatible list:

curl https://gateway.smith.langchain.com/v1/models \
    -H "Authorization: Bearer $LANGSMITH_API_KEY"

{
  "object": "list",
  "data": [
    {"id": "openai/gpt-5.4-mini", "object": "model"},
    {"id": "fireworks/accounts/fireworks/models/glm-5p2", "object": "model"},
    {"id": "anthropic/claude-sonnet-4-6", "object": "model"}
  ]
}

Every model ID is prefixed with the provider name in the form <provider>/<model>; use this prefixed ID as model when making a call. A provider without a configured secret is omitted.

Use a regional gateway

Replace gateway.smith.langchain.com with the hostname for your LangSmith region:

Region	Gateway hostname
GCP US	`gateway.smith.langchain.com`
GCP EU	`eu.gateway.smith.langchain.com`
GCP APAC	`apac.gateway.smith.langchain.com`
AWS US	`aws.gateway.smith.langchain.com`

Keep the same path for the selected API format.

Handle errors

Status or symptom	Meaning
`400 Bad Request`	The request is malformed, the model ID is not provider-prefixed, or the request cannot be translated.
`401 Unauthorized`	The LangSmith API key is missing or invalid.
`403 Forbidden`	The key does not have the required gateway permissions.
`429 Too Many Requests`	A gateway rate limit or an upstream provider rate limit was reached.
No models with a provider prefix appear in `GET /v1/models`	The provider may not be configured or may not have returned a model catalog.

For setup-specific resolutions, see the Quickstart.

Core capabilities

Administration and governance

Advanced

Compare API formats

Use Chat Completions

Use Messages

Use Responses

Understand translation behavior

List models

Use a regional gateway

Handle errors

See also

​Compare API formats

​Use Chat Completions

​Use Messages

​Use Responses

​Understand translation behavior

​List models

​Use a regional gateway

​Handle errors

​See also

Compare API formats

Use Chat Completions

Use Messages

Use Responses

Understand translation behavior

List models

Use a regional gateway

Handle errors

See also