Per-customer policies - Docs by LangChain

Private beta: The LLM Gateway is in private beta. Sign up for the waitlist to get access.

A spend policy or rate limit policy can carry a condition on a custom request header, so traffic from a single subject splits into separate limits by header value. Use this to cap each of your own end customers, tenants, or teams without issuing a separate LangSmith API key for each one. For example, a policy scoped to a workspace with the condition X-Gateway-Customer-Id: acme limits only the requests from that workspace that carry that header value. Requests from the same workspace carrying X-Gateway-Customer-Id: globex count against a different policy.

Matchable headers

The gateway matches on request headers prefixed with X-Gateway-, and on keys inside the X-Gateway-Metadata JSON header. No other request header is matchable. Header names are normalized before matching: the X-Gateway- prefix is stripped, the remainder is lowercased, and every character outside a-z, 0-9, and _ is replaced with _. The headers X-Gateway-Customer-Id, x-gateway-customer_id, and X-Gateway-CUSTOMER.ID all resolve to the matcher key customer_id. Header values are compared as exact, case-sensitive strings, with no wildcard or pattern matching. The gateway stamps caller identity itself and ignores client attempts to override it. Headers that resolve to organization_id, workspace_id, workspace_handle, user_id, user_email, api_key_id, api_key_short, auth_mode, user_agent, applied_policy_ids, or applied_policy_names, and any header whose normalized name starts with gateway, are discarded.

Header condition rules

One condition per policy: A policy accepts a single header key with a single value.
Pairs with one subject scope: Combine a header condition with an organization, workspace, user, or API key scope. The subject side accepts several values and matches any of them. The header side accepts exactly one value.
Spend caps and rate limits only: Default policies cannot carry a header condition, so the gateway never creates per-header policies on its own. To limit many header values, create one policy for each.
A missing header matches nothing: A request that does not carry the header does not match the policy. Pair per-header policies with a broader policy on the subject itself so untagged traffic is still limited.
Every matching policy is enforced: A request that matches both a plain subject policy and a policy with a header condition counts against both, and either one can block it.
At most 10 conditions: A policy carries no more than 10 subject conditions in total.

Add a header condition

Creating and managing policies requires the organization:manage permission. For the full permissions breakdown, refer to Traces, Engine, and access control.

Go to Settings → Gateway → LLM Gateway.
Click Create policy.
Select the policy type and subject scope, then set the limits.
Under Custom header condition (optional), enter the Header name without its X-Gateway- prefix (for example, Customer-Id) and the Header value to match (for example, acme).
Save.

You cannot edit the header condition in the UI after the policy is created. To change it, delete the policy and create a new one, or update subject_matchers through the API.

Cap spend per end customer

A reseller or multi-tenant application usually calls the gateway from its own backend, using one workspace-scoped API key on behalf of many end customers. Header conditions give each of those end customers a separate cap under that single key.

The gateway trusts the X-Gateway-* headers on an incoming request. Set the header in your own backend after you authenticate the end user, and do not distribute the gateway API key to end users. A caller that controls both the key and the header can choose which cap to spend against.

Step 1. Send a customer header on every call

Attach the header to each request your backend makes on behalf of an end customer:

curl https://gateway.smith.langchain.com/openai/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -H "X-Gateway-Customer-Id: acme" \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}'

import os

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["LANGSMITH_API_KEY"],
)
customer_id = "acme"
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "ping"}],
    extra_headers={"X-Gateway-Customer-Id": customer_id},
)
print(response.choices[0].message.content)

If your LangSmith account is on a regional instance, use the corresponding regional gateway.

Step 2. Create a cap for one customer

Create one spend policy per end customer through the LangSmith REST API:

curl -X POST "https://api.smith.langchain.com/v1/platform/gateway-policies" \
    -H "X-Api-Key: $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
          "name": "customer-acme-monthly-cap",
          "policy_type": "spend_cap",
          "action": "block",
          "subject_matchers": [
            {"key": "workspace_id", "value": "0b1c2d3e-4f56-7890-abcd-ef1234567890"},
            {"key": "customer_id", "value": "acme"}
          ],
          "config": {"window": "monthly", "limit_usd": 250}
        }'

import os

import httpx

response = httpx.post(
    "https://api.smith.langchain.com/v1/platform/gateway-policies",
    headers={"X-Api-Key": os.environ["LANGSMITH_API_KEY"]},
    json={
        "name": "customer-acme-monthly-cap",
        "policy_type": "spend_cap",
        "action": "block",
        "subject_matchers": [
            {"key": "workspace_id", "value": "0b1c2d3e-4f56-7890-abcd-ef1234567890"},
            {"key": "customer_id", "value": "acme"},
        ],
        "config": {"window": "monthly", "limit_usd": 250},
    },
    timeout=30.0,
)
response.raise_for_status()

The matcher key is the normalized name customer_id, not the header name X-Gateway-Customer-Id. The policy belongs to the organization that owns the API key. Posting a policy whose subject_matchers already exist updates that policy instead of adding a duplicate, so this call is safe to repeat.

Step 3. Sync policies with your customer list

Because each end customer needs its own policy, keep the policy set in step with your customer list. The following script creates or updates a cap for every current customer, then deletes the caps of customers that are gone:

import os

import httpx

API_URL = "https://api.smith.langchain.com/v1/platform/gateway-policies"
WORKSPACE_ID = os.environ["LANGSMITH_WORKSPACE_ID"]

# Your source of truth: end customer identifier mapped to a monthly cap in USD.
CUSTOMER_CAPS = {"acme": 250.0, "globex": 1000.0, "initech": 50.0}


def matchers_for(customer: str) -> list[dict[str, str]]:
    return [
        {"key": "workspace_id", "value": WORKSPACE_ID},
        {"key": "customer_id", "value": customer},
    ]


def existing_caps(client: httpx.Client) -> dict[str, dict]:
    """Return the current per-customer spend caps, keyed by customer identifier."""
    response = client.get(API_URL, params={"policy_type": "spend_cap"})
    response.raise_for_status()
    return {
        matcher["value"]: policy
        for policy in response.json()
        for matcher in policy["subject_matchers"]
        if matcher["key"] == "customer_id"
    }


def sync() -> None:
    headers = {"X-Api-Key": os.environ["LANGSMITH_API_KEY"]}
    with httpx.Client(headers=headers, timeout=30.0) as client:
        existing = existing_caps(client)

        # Posting an existing matcher set updates that policy, so this both
        # creates caps for new customers and corrects caps that changed.
        for customer, limit_usd in CUSTOMER_CAPS.items():
            client.post(
                API_URL,
                json={
                    "name": f"customer-{customer}-monthly-cap",
                    "policy_type": "spend_cap",
                    "action": "block",
                    "subject_matchers": matchers_for(customer),
                    "config": {"window": "monthly", "limit_usd": limit_usd},
                },
            ).raise_for_status()

        # Deletes any per-customer cap missing from CUSTOMER_CAPS, including
        # caps created outside this script.
        for customer, policy in existing.items():
            if customer not in CUSTOMER_CAPS:
                client.delete(f"{API_URL}/{policy['id']}").raise_for_status()


if __name__ == "__main__":
    sync()

Run the script whenever a customer signs up, churns, or moves to a different plan.

Step 4. Read spend per customer

Each spend policy returned by the API reports current_spend_usd, the spend accumulated in the policy’s active window. Use it to show each end customer their usage, or to warn them before they reach the cap. The field is omitted when the spend lookup fails, so treat a missing value as unknown rather than as zero. The list endpoint narrows by a subject matcher key only when that key is paired with a value, so list the spend caps and select the per-customer ones in your own code:

import os

import httpx

response = httpx.get(
    "https://api.smith.langchain.com/v1/platform/gateway-policies",
    headers={"X-Api-Key": os.environ["LANGSMITH_API_KEY"]},
    params={"policy_type": "spend_cap"},
    timeout=30.0,
)
response.raise_for_status()

for policy in response.json():
    customer = next(
        (m["value"] for m in policy["subject_matchers"] if m["key"] == "customer_id"),
        None,
    )
    if customer is None:
        continue  # A cap on the workspace itself, not on one end customer.
    # current_spend_usd is absent when the spend lookup fails.
    print(customer, policy.get("current_spend_usd"), policy["config"]["limit_usd"])

Limit throughput per end customer

Rate limits use the same subject matchers. Swap policy_type and config to give an end customer its own request and token allowance:

curl -X POST "https://api.smith.langchain.com/v1/platform/gateway-policies" \
    -H "X-Api-Key: $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
          "name": "customer-acme-rate-limit",
          "policy_type": "rate_limit",
          "action": "block",
          "subject_matchers": [
            {"key": "workspace_id", "value": "0b1c2d3e-4f56-7890-abcd-ef1234567890"},
            {"key": "customer_id", "value": "acme"}
          ],
          "config": {
            "version": 1,
            "limits": [
              {"metric": "requests", "window": "minute", "value": 100},
              {"metric": "tokens", "window": "hour", "value": 1000000}
            ]
          }
        }'

The sync script in step 3 applies to rate limits with the same two substitutions. Spend caps and rate limits are separate families, so an end customer can hold one of each on the same header value.

Next steps

Spend policies: set cost caps for organizations, workspaces, users, and API keys.
Rate limit policies: limit requests and tokens in a rolling window.
Traces and access control: understand where gateway traces land and who can configure policies.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

​Matchable headers

​Header condition rules

​Add a header condition

​Cap spend per end customer

​Step 1. Send a customer header on every call

​Step 2. Create a cap for one customer

​Step 3. Sync policies with your customer list

​Step 4. Read spend per customer

​Limit throughput per end customer

​Next steps

Matchable headers

Header condition rules

Add a header condition

Cap spend per end customer

Step 1. Send a customer header on every call

Step 2. Create a cap for one customer

Step 3. Sync policies with your customer list

Step 4. Read spend per customer

Limit throughput per end customer

Next steps