Skip to main content
The LLM Gateway is in beta.
The LLM Gateway is the control layer between the agents and applications you run and the model providers your workspace has configured. Because the gateway is part of LangSmith, each call it routes becomes a LangSmith trace that records the routing and policy decisions made for it. Your application authenticates with a LangSmith API key. The gateway routes each request to the provider named in its model ID, traces the call, and applies your organization’s governance policies. Without the gateway, each application and deployment holds its own provider keys. Reliability, spend, and data controls are set up separately for each provider account. With the gateway, provider keys stay in LangSmith, and model fallbacks, spend limits, rate limits, and data policies apply to every call in their scope. When a fallback fires or a policy blocks a call, the trace shows each attempt and policy check. Moreover, policy violations surface in LangSmith Engine for triage.

​
Choose a request format

The gateway accepts OpenAI Chat Completions, Anthropic Messages, and OpenAI Responses requests. The request format and the provider are independent: a Chat Completions request can call an Anthropic model, and the gateway translates the request and response when the provider uses a different native format. Keep the format your application already uses, and switch providers by changing the model ID. For endpoints, model ID syntax, and translation behavior, see API formats. LangChain chat models and Deep Agents reach the gateway through the LANGSMITH_GATEWAY environment variable, which uses direct model access in each provider’s native format, so no translation applies.

​
Choose how credentials are managed

The gateway calls each provider with a credential that LangSmith holds: your own provider key, stored as a workspace secret, or Gateway Credits, which LangChain bills to your LangSmith account. For every credential source and who bills it, see Provider authentication.

​
Next steps

Start with the guide for your role:

Set up the gateway for your organization

Complete the initial setup to route requests through the gateway.

Send your first request

Send a request from the Simulator, call the gateway from your code, and view the trace.
For further questions, contact LangChain support.
Connect these docs to your agent of choice via MCP for real-time answers.
Edit this page on GitHub or file an issue.