The LLM Gateway is in beta.
gateway.request root run. Policy violations surface in LangSmith Engine for triage.
Find gateway traces
Every gateway request is traced to a project namedgateway in the workspace associated with the caller’s API key. To view gateway requests, go to LLM Gateway and select the Traces tab. The Simulator’s View trace button opens its request in this tab. To see the requests behind one user’s or API key’s spend, select that user or key in the Usage tab, and click View traces. On the Developer plan, open the gateway tracing project in your workspace instead.
Trace content (request and response bodies) is off by default for all organizations, so gateway traces record only metadata, such as token counts, policy outcomes, and caller identity. These traces have ls_metadata_only set to true. Traces without content are not billed against your LangSmith trace quota. To store content, turn on Trace content in a data policy. A content-logged request counts once toward your plan’s trace volume, regardless of how many provider attempts it makes.
Any user who can view runs in the workspace can view its gateway traces. To limit access, see Restrict trace visibility.
Read the trace tree
Expandgateway.request to inspect each attempt. Each attempt holds its own policy checks and provider call, so a request that falls back repeats the structure below once per attempt:
gateway. in each run’s metadata, recorded as langsmith.metadata.gateway.* span attributes. The gateway sets these keys itself, so clients cannot override them through request headers. Each run below lists the keys most worth checking:
gateway.request: The root run, which shows what happened to the request.gateway.request.outcomeissuccess,upstream_error,blocked,rejected, orfallback_exhausted.gateway.request.error_reasonexplains a blocked or rejected request. A throttled request has outcomeblockedand error reasonrate_limit, not a separatethrottledoutcome.gateway.request.served_modelnames the model that answered.gateway.request.idcorrelates the root, attempt, policy, and provider runs for the request, including in bulk exports.
gateway.attempt #N: A model candidate: the primary, a fallback, or a/routescandidate. An attempt does not always make a provider call.gateway.attempt.outcomerecords its result. A candidate skipped for cooldown has outcomeskippedand no provider LLM run.skippedis an attempt-only outcome, not a request outcome.gateway.attempt.upstream_status_coderecords the provider’s status code when the call failed.
gateway.policy_check: The check against spend, rate limit, model access, and routing policies. This run shows that the gateway evaluated policies for the attempt. To keep the trace uncluttered, only a policy that denied or throttled the request appears as apolicy: <name>child run.gateway.policy.action, on eachpolicy: <name>run, records whether the policy denied or throttled the request.gateway.policy.matched_nameslists all matched policy names, including policies that did not deny or throttle the request.
gateway.data_policy: The data policy scan for PII and secrets, which runs separately from the policy check. A data policy that redacted or blocked content appears as apolicy: <name>child run, with arule: <rule_id>run for each rule that matched.gateway.rule.matches, on eachrule: <rule_id>run, counts how many times the rule matched.
- Provider LLM run: The call to the provider, when one was made. Select it for token usage, cost, and, when trace content is on, the request and response.
ls_model_nameandls_provideridentify the model and provider. These keys are on the provider LLM run, not the root run.gateway.credential_sourcerecords which credential paid for the call, such asbyokfor the workspace’s provider key orcreditsfor Gateway Credits.
A blocked attempt can have policy runs without a provider LLM run. A request rejected before any attempt has only the root run.
Filter traces by caller
The root run and provider LLM runs record who made the request. The keys below are the most useful for finding one caller’s traces:gateway.api_key_id: The API key that made the request.gateway.user_id: The user who made the request. Set only for personal access tokens, so service-key requests have no user.gateway.<name>: EachX-Gateway-*request header. For how names are normalized, see Matchable headers.
Restrict trace visibility
Gateway traces are runs in a workspace project, so the same permissions govern them as other runs in the workspace. Any user withruns:read (and projects:read to see the project itself) on the workspace can view traces in that workspace’s gateway project. The built-in Workspace Admin, Workspace Editor, and Workspace Viewer roles all include both permissions by default.
You have two options to limit access to gateway traces:
- Separate workspaces (works on any plan): create one workspace with restricted membership and another for developer coding agents with broader membership. Each workspace has its own provider secrets and trace projects.
- Project-level access policies (requires an Enterprise plan): write an ABAC policy restricting
projects:readandruns:readon the gateway project to specific users or roles.
Triage policy violations
When a governance policy fires (such as when a spend limit is hit, PII is redacted, or a secret is caught), the event is recorded as metadata on the trace. These policy violations surface as issues in LangSmith Engine. From an Engine issue, you can:- See the violation: find which policy fired and what it blocked or redacted.
- Click through to the trace: see exactly what the agent was doing when the policy triggered.
- Diagnose the root cause: such as a retry loop burning budget, a user pasting credentials into a prompt, or a legitimate workload that outgrew its cap.
- Take action: update the agent’s configuration, adjust the policy, or escalate.
See also
- Monitor LLM Gateway spend: track spend by user, API key, model, and provider.
- Spend policies: attach cost limits to API keys and users.
- Data policies: configure data policies.
Connect these docs to your agent of choice via MCP for real-time answers.

