API formats

​

Compare API formats

​

Understand translation behavior

• When the provider supports the selected format natively, the gateway preserves that format.
• Otherwise, the gateway translates the request into a format supported by the provider and translates the response back, including streaming responses.
• Translation can reject fields that cannot be represented in the target provider format. Use direct model access when provider-native behavior is required.

​

Use prompt caching

• OpenAI models: OpenAI caches prompts automatically. To control cache lifetime or breakpoints, use OpenAI’s caching fields, such as prompt_cache_options or prompt_cache_retention. Which fields and values each model accepts is set by OpenAI. For details, see the OpenAI prompt caching guide.
• Anthropic models in Messages format: Use Anthropic’s cache_control field. For details, see the Anthropic prompt caching guide.
• Anthropic models in Chat Completions or Responses format: Set any of prompt_cache_options, prompt_cache_retention, or prompt_cache_key. The gateway turns on Anthropic prompt caching with Anthropic’s default five-minute cache lifetime. It does not carry over a requested lifetime, retention, or breakpoint mode.
• OpenAI models in Messages format: The gateway drops cache_control fields. OpenAI’s automatic caching still applies.

​

Send a request

curl https://gateway.smith.langchain.com/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"<MODEL_ID>","messages":[{"role":"user","content":"Hello!"}]}'

curl https://gateway.smith.langchain.com/v1/messages \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"<MODEL_ID>","max_tokens":1024,"messages":[{"role":"user","content":"Hello!"}]}'

curl https://gateway.smith.langchain.com/v1/responses \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"<MODEL_ID>","input":"Hello!"}'

​

Choose a model ID

• Your own provider key: <provider>/<model>, such as openai/gpt-5.4-mini or azure/<deployment-name>. The gateway calls the provider with the provider key stored in your workspace.
• Gateway Credits: a hosted model slug, such as moonshotai/kimi-k3. Gateway Credits bill LangChain-hosted models to your LangSmith account, with no provider key required.
• Saved model configuration: custom/<name>. The gateway uses the provider, model, and credentials from the saved model configuration.

​

List models

curl https://gateway.smith.langchain.com/v1/models \
    -H "Authorization: Bearer $LANGSMITH_API_KEY"

{
  "object": "list",
  "data": [
    {"id": "openai/gpt-5.4-mini", "object": "model"},
    {"id": "fireworks/accounts/fireworks/models/glm-5p2", "object": "model"},
    {"id": "anthropic/claude-opus-5", "object": "model"},
    {"id": "moonshotai/kimi-k3", "object": "model"}
  ]
}

​

Read response metadata

​

Fix request errors

​

See also

• Send your first LLM Gateway request: make your first request and view its trace.
• How the gateway works: what happens to each request, and which hostname to use in each region and on BYOC.
• Direct model access: bypass format translation and use provider-native APIs.
• Model fallbacks: retry requests against backup models.