Skip to main content
The LLM Gateway is in beta.
Every call through the LLM Gateway is traced to LangSmith, and policy violations surface in LangSmith Engine for triage.

​
Where gateway traces appear

Every gateway-proxied call is traced to a project named gateway in the workspace associated with the caller’s API key. To control who can see these traces, see Restrict trace visibility.

​
Open the Traces tab

To view gateway requests, go to LLM Gateway and select the Traces tab. The Simulator’s View trace button opens its request in this tab, and the View traces button in a Usage detailed view opens the traces behind one user’s, API key’s, or header value’s spend. On the Developer plan, open the gateway tracing project in your workspace instead. Trace content (request and response bodies) is off by default for all organizations, so gateway traces record only metadata, such as token counts, policy outcomes, and caller identity. Traces without content are not billed against your LangSmith trace quota. To store content, turn on Trace content in a data policy. Traces that match a content-logging data policy count toward your plan’s trace volume.

​
Trace metadata

Gateway-proxied calls are distinguishable from direct LLM calls by the project they land in and the metadata attached to their spans. Gateway-specific keys appear under gateway. in the trace metadata, recorded as langsmith.metadata.gateway.* span attributes:
  • Gateway project: Filter by the gateway project, or by the presence of langsmith.metadata.gateway.* span attributes, to find gateway-proxied calls.
  • Caller identity: gateway.workspace_id, gateway.api_key_id, and gateway.user_agent, the calling client’s User-Agent header. gateway.user_id is set only for personal access tokens, so service-key calls have no user.
  • Credential source: gateway.credential_source names the credential that paid for the provider call: byok for the workspace’s provider key, credits for Gateway Credits, subscription_oauth for a Claude subscription, or platform_managed. Every gateway trace carries this key.
  • Authentication mode: gateway.auth_mode is set only when the caller did not authenticate with a LangSmith API key, such as claude_max_oauth for a Claude subscription.
  • Model and provider: ls_model_name and ls_provider, which the Usage tab groups spend by.
  • Request headers: each X-Gateway-* request header, recorded as gateway.<name>. For how names are normalized, see Matchable headers.
  • Policy evaluation results: Every gateway span records which policies were evaluated and their outcome via langsmith.metadata.gateway.policy.matched_ids/_names, passed_ids/_names, and violated_ids/_names, so both passes and blocks are captured.
  • Guard rule matches: When redaction policies apply, the guard pipeline emits a rule_id → count map stamped onto the span as policy.matched_rules, passed_rules, and violated_rules. These are rule IDs, not PII or secret category labels.
  • Content flags: ls_metadata_only is true on a trace stored without request and response content. content_quota_exceeded is also true when the gateway dropped the content because a monthly trace limit was reached.
  • Cost data: Token counts and cost are computed inline and feed the same spend accumulator that spend-cap policies enforce against.
The gateway sets these keys itself. Clients cannot override them through request headers.

​
Restrict trace visibility

Gateway traces are written as runs into a workspace project and follow LangSmith’s standard workspace membership model. Anyone with runs:read (and projects:read to see the project itself) on the workspace can view traces in that workspace’s gateway project. The built-in Workspace Admin, Workspace Editor, and Workspace Viewer roles all include both permissions by default. If you need to limit who can see gateway traces, you have two options:
  • Separate workspaces (works on any plan): create one workspace with restricted membership and another for developer coding agents with broader membership. Each workspace has its own provider secrets and trace projects.
  • Project-level access policies (requires an Enterprise plan): write an ABAC policy restricting projects:read and runs:read on the gateway project to specific users or roles.

​
LangSmith Engine integration

When a governance policy fires (such as when a spend limit is hit, PII is detected and redacted, or a secret is caught), the event is recorded as metadata on the trace. These policy violations surface as issues in LangSmith Engine. From an Engine issue, you can:
  1. See the violation: which policy fired, what was blocked or redacted.
  2. Click through to the trace: see exactly what the agent was doing when the policy triggered.
  3. Diagnose the root cause: was it a retry loop burning budget, a user pasting credentials into a prompt, or a legitimate workload that outgrew its cap, etc.
  4. Take action: update the agent’s configuration, adjust the policy, or escalate.

​
Next steps

Connect these docs to your agent of choice via MCP for real-time answers.
Edit this page on GitHub or file an issue.