Set up the LLM Gateway

This page walks an admin through preparing the LLM Gateway for their organization: connecting providers, granting access, issuing API keys, and setting policies. Complete the following sections in order. Each one links to the page that covers its topic in full.

Enable the gateway on self-hosted LangSmith

On self-hosted LangSmith, enable the gateway in the Helm chart before the other steps. Skip this section on other deployments.

Add provider keys

Adding a provider key requires permission to manage secrets in that workspace. The gateway calls each provider with a key stored as a secret in the workspace that the caller’s API key belongs to. Add a key for each provider you route requests to, in each workspace whose API keys send those requests. The gateway does not fall back to organization secrets for these calls. Skip this section if you route requests only to Gateway Credits models, which need no provider key. For the steps and the secret name each provider uses, see Add a provider key.

Grant permission to call the gateway

An API key needs the gateway:invoke and workspaces:read permissions to call the gateway. On plans without workspace roles, every user has the Workspace Admin role, which includes both, so no action is needed. On the Enterprise plan, the built-in Workspace Editor and Workspace Viewer roles include neither permission. As an Organization Admin, create a custom role with both permissions and assign it to the users who call the gateway, or assign them the Workspace Admin role. For more information, see Grant permission to call the gateway.

Decide how developers get API keys

Each request authenticates with a workspace-scoped LangSmith API key. Developers can create their own personal access token (PAT), which carries their role, or the Simulator creates one for them. For shared agents or a company-wide rollout through mobile device management (MDM), you can create service keys instead. The key type also determines how the Usage tab can attribute spend. Requests made with a personal access token count toward a user, while requests made with a service key count only toward the key. If you plan to charge back LLM costs, see Attribute spend for chargebacks before you issue keys. For key types and scoping, see Choose a key type. For coding agent rollouts, see Set up coding agents.

Optional: Set policies

Creating policies requires the organization:gateway:manage permission, which Organization Admins and Organization Operators have by default. On self-hosted LangSmith earlier than application version 0.18.9 (Helm chart versions before 0.18.0-rc.22), it requires organization:manage instead. Use the tabs on the LLM Gateway page to create any of these policy types:

• Spend limits: Hard caps at the organization, workspace, API key, or user level. Your organization may already have a default limit on the Cost Controls tab.
• Rate limits: Request or token limits in a time window.
• Model access: Which models users can call.
• Header-based policies: Spend caps, rate limits, and model access scoped by a custom request header, such as a customer, tenant, or cost center identifier.
• Data policies: Detect and redact PII and secrets before they reach the model, and control whether request and response bodies are traced.

On the Enterprise plan, creating, updating, and deleting policies is recorded in audit logs.

Send a test request

To confirm the setup, follow Send your first LLM Gateway request. It sends a request from the Simulator and from code, and shows its trace. Share the same page with developers as their starting guide.

---