Enable the LLM Gateway

​

Enable the gateway

agentGateway:
  enabled: true

  # Defaults shown for reference—override only if you have a reason.
  # containerPort: 8083
  # deployment:
  #   replicas: 1
  #   resources:
  #     limits:   { cpu: 2, memory: 2Gi }
  #     requests: { cpu: 1, memory: 1Gi }
  # autoscaling:
  #   enabled: true
  #   minReplicas: 1
  #   maxReplicas: 5
  #   targetCPUUtilizationPercentage: 50
  #   targetMemoryUtilizationPercentage: 70
  # service:
  #   type: ClusterIP
  #   port: 8083

helm search repo langchain/langsmith --versions

helm upgrade --install langsmith langchain/langsmith \
  --version <chart-version> \
  -f values.yaml --timeout=20m

kubectl get deploy langsmith-agent-gateway
kubectl rollout status deploy/langsmith-agent-gateway

​

Enable Presidio PII detection (optional)

presidioAnalyzer:
  enabled: true

​

Expose the gateway externally

curl -sS -o /dev/null -w '%{http_code}\n' https://<your-hostname>/gateway/health
# Expected: 200

​

Roll back

agentGateway:
  enabled: false

​

Make your first call

# Replace with your Ingress hostname.
export LANGSMITH_GATEWAY_BASE_URL="https://<your-hostname>/gateway/v1"
export LANGSMITH_API_KEY="lsv2_..."

export LANGSMITH_GATEWAY="https://<your-hostname>/gateway"
export LANGSMITH_GATEWAY_API_KEY="$LANGSMITH_API_KEY"

curl "$LANGSMITH_GATEWAY_BASE_URL/chat/completions" \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}'

import os

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LANGSMITH_GATEWAY_BASE_URL"],
    api_key=os.environ["LANGSMITH_API_KEY"],
)
response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "ping"}],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.LANGSMITH_GATEWAY_BASE_URL,
  apiKey: process.env.LANGSMITH_API_KEY,
});
const response = await client.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "ping" }],
});
console.log(response.choices[0].message.content);

import os

from langchain.agents import create_agent
from langchain.chat_models import init_chat_model

model = init_chat_model(
    model="openai/gpt-4o-mini",
    model_provider="openai",
    base_url=os.environ["LANGSMITH_GATEWAY_BASE_URL"],
    api_key=os.environ["LANGSMITH_API_KEY"],
)
agent = create_agent(model=model, system_prompt="You are a helpful assistant.")
result = agent.invoke({"messages": [{"role": "user", "content": "ping"}]})
print(result["messages"][-1].content)

{"type": "error", "error": {"message": "request blocked by gateway policies: R&D Spend Cap"}}

​

Troubleshooting

• Gateway pod CrashLoopBackOff immediately after upgrade: Check kubectl logs deploy/langsmith-agent-gateway.
• Model calls fail with connection refused or timeout to agent-gateway: Verify the gateway pod is Running and that no NetworkPolicy in your cluster blocks port 8083 between pods in the LangSmith namespace.

​

Security notes

• The gateway makes outbound HTTP calls to provider LLM endpoints. Any custom provider URLs you configure must be validated against your egress policy; the service uses LangSmith’s SSRF-protection library for custom and configured provider URLs. Built-in providers resolve to fixed, provider-owned hostnames and aren’t a user-controlled-URL path.
• External clients should reach the gateway through the frontend’s /gateway/ path so requests inherit the same TLS termination, auth, and rate limiting as the rest of the LangSmith API. Creating a second Ingress or LoadBalancer directly to port 8083 bypasses those controls.
• AGENT_GATEWAY_URL is derived from the Helm release fullname and the chart’s agentGateway.name, agentGateway.service.port, namespace, and clusterDomain values. Overriding any of those on an existing install requires a rolling restart of every pod that reads the value.

​

Next steps

• Quickstart: the SaaS equivalent of the steps above.
• Admin setup: grant workspace users gateway access and configure provider secrets.
• Spend policies: cap spend at the organization, workspace, API key, or user level.