Documentation Index
Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
Use this file to discover all available pages before exploring further.
Private beta: The LLM Gateway is in private beta. Sign up for the waitlist to get access.
402 response:
Policy dimensions
Spend policies are evaluated from broadest to most specific. All matching policies are checked, and if any one returns a block, the request is rejected. You can set a policy as a default (applying a blanket spend cap to all workspaces, users, or API keys) or as a granular policy (individual limits or limits on a group of entities).| Scope | What it caps | Example |
|---|---|---|
| Organization | Total spend across all workspaces in the org | ”The entire org cannot spend more than $10,000/month on LLM calls” |
| Workspace | Total spend within a single workspace or group of workspaces | ”The workspaces related to R&D cannot spend more than $2,000/month” |
| API key | Spend by a single API key or group of API keys (maps to a service or agent) | “The customer support agent keys cannot spend more than $500/month cumulatively” |
| User | Spend by a single user or group of users (resolved from the API key’s identity) | “No individual developer can spend more than $50/day” |
Conflict resolution
By default, LLM Gateway assesses the broadest scope first. If a granular policy applies, the most restrictive policy wins. Narrower scopes can only tighten limits, never loosen them. If an org-level policy caps spend at $10,000/month and a workspace-level policy caps at $15,000/month, the $10,000 org cap still applies.Defaults vs. granular policies
Spend policies have two aspects:- Sums across a dimension: the total cap for that scope. Example: “This workspace’s total spend cannot exceed $5,000/month.”
- Defaults for each member of a dimension: a base limit that applies to every API key or user within a scope unless overridden. Example: “Each API key in this workspace gets a $200/month default cap.” Individual API keys can receive additional policies that raise their specific limit, but no policy can loosen a cap set at a broader scope.
Time windows
| Window | Resets | Use case |
|---|---|---|
| Monthly | First of each month | Budget alignment, overall cost control |
| Weekly | Midnight UTC on the Monday of each week | weekly budgeting |
| Daily | Midnight UTC | Prevent single-day cost spikes (for example, a coding agent in a retry loop overnight) |
| Hourly | Top of each hour | Catch runaway agents quickly |
Create a spend policy
- Go to Settings → Gateway → LLM Gateway.
- Click Create policy.
- Select the scope (organization, workspace, API key, or user).
- Set the time window (monthly, weekly, daily, or hourly).
- Set the spend cap in USD.
- Save.
View spend
The spend visibility dashboard shows real-time cost rollups so you can understand where your LLM budget is going before you reach the limit. From the gateway settings page, you can view how much each policy has spent against its cap.Integration with LangSmith Engine
When a spend policy blocks a request, the violation is recorded as metadata on the trace. These violations surface as issues in LangSmith Engine, where you can click through from the issue to the trace to understand what the agent was doing when it hit the limit. This is useful for diagnosing whether a blocked request represents a genuine cost problem (a coding agent in a retry loop) or a policy that needs adjustment (a legitimate workload that grew beyond its cap).Next steps
- PII and secrets redaction: add data protection policies alongside cost controls.
- Traces, Engine, and access control: how policy events flow into observability and triage.
Connect these docs to Claude, VSCode, and more via MCP for real-time answers.