> ## Documentation Index
> Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Model providers
The [Playground](/langsmith/prompt-engineering-concepts#playground) supports a wide range of model providers. You can select a provider, configure your preferred settings, and save these configurations to reuse across multiple prompts.
Use this page for a list of the available providers and their configuration options:
For details on creating and managing model configurations, refer to the [Configure prompt settings](/langsmith/managing-model-configurations) page.
## Amazon Bedrock
Before you use this model, ensure you have [AWS credentials or IAM role](https://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html).
### Authentication
Amazon Bedrock supports two authentication methods. **IAM trusted entity is the recommended approach** because it avoids sharing long-lived AWS access keys with LangSmith.
#### IAM trusted entity (recommended)
With IAM trusted entity authentication, you create an IAM role in your AWS account and allow LangSmith to assume it. No access keys are stored in LangSmith. Instead, LangSmith uses [AWS STS](https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html) to assume the role on each request.
To set this up:
1. Create an IAM role in your AWS account with permissions to invoke Bedrock models (e.g., `bedrock:InvokeModel`).
2. Add a trust policy that allows LangSmith's AWS account (`808407022534`) to assume the role, using your LangSmith workspace ID as the external ID:
```json theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::808407022534:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": ""
}
}
}
]
}
```
You can find your workspace ID in your [LangSmith workspace settings](https://smith.langchain.com/settings).
3. In the LangSmith Playground, expand the **IAM Trusted Entity** section under the Bedrock provider and enter the ARN of the role you created.
For more details on trust policies, see the [AWS documentation](https://aws.amazon.com/blogs/security/how-to-use-trust-policies-with-iam-roles/).
#### Access keys
Alternatively, you can authenticate with AWS access keys (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`). Enter these in the Bedrock provider configuration in the Playground. This method is simpler to set up but less secure because it requires storing long-lived credentials.
### Available models
AWS Bedrock provides access to foundation models from multiple providers:
* **Anthropic:** Claude models.
* **Amazon:** Titan models.
* **Cohere:** Command models.
* **Meta:** Llama models.
* **Others:** Additional providers available based on region.
For the current list of available models, refer to the [AWS Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html).
### Configuration parameters
Parameters depend on the underlying model provider:
#### For Anthropic models
Uses Anthropic configuration (see [Anthropic](#anthropic) section below).
#### For Amazon Titan
| Parameter | Range | Description |
| --------------- | --------- | ----------------------- |
| **Temperature** | 0.0 - 1.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
#### AWS-specific settings
* **Region:** AWS region for model deployment.
### Tool calling
Depends on underlying model:
* **Anthropic models:** `auto`, `any`.
* **Cohere models:** `auto`.
## Anthropic
Before you use this model, ensure you have an [Anthropic API key](https://console.anthropic.com/settings/keys).
### Available models
Anthropic offers three tiers of models across their Claude generations:
* **Opus:** Highest intelligence and capability.
* **Sonnet:** Balanced performance and cost.
* **Haiku:** Fast and cost-effective.
Recent Claude models support extended thinking capabilities for showing reasoning processes.
For the current list of available models, refer to the [Anthropic documentation](https://docs.anthropic.com/claude/docs/models-overview).
### Configuration parameters
| Parameter | Range | Default | Description |
| --------------------- | --------- | -------- | -------------------------------------------------- |
| **Temperature** | 0.0 - 1.0 | Optional | Randomness control (uncheck to use model default) |
| **Max Output Tokens** | 1+ | 1024 | Maximum response length |
| **Top P** | 0.0 - 1.0 | Optional | Nucleus sampling (uncheck for model default) |
| **Top K** | 1+ | Optional | Limits to top K tokens (uncheck for model default) |
Temperature, Top P, and Top K are optional. When unchecked, Claude uses its internal defaults.
#### Extended Thinking
Available on supported Claude models. Enable the model to show reasoning before responding, similar to OpenAI's o-series.
| Parameter | Range | Description |
| ---------------------------- | ------ | --------------------------------------- |
| **Enable Extended Thinking** | Toggle | Show/hide thinking process |
| **Budget Tokens** | 1+ | Max tokens for thinking (default: 1024) |
When enabled, responses include:
1. A "thinking" section with the model's reasoning.
2. The final response.
#### Advanced options
* **Base URL:** Override API endpoint for custom deployments.
### Tool calling
* **Supported Tool Choices:** `auto`, `any` (requires at least one tool).
* **Parallel Execution:** No (sequential only).
## Azure OpenAI
Before you use this model, ensure you have [Azure OpenAI credentials](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart) (endpoint + API key).
### Available models
Azure OpenAI provides the same model families as OpenAI:
* **GPT series:** General-purpose chat models.
* **o-series:** Reasoning-focused models.
* **Legacy models:** GPT-3.5 and GPT-4 variants.
Model availability varies by Azure region and requires deployment before use.
For the current list of available models, refer to the [Azure OpenAI documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).
### Configuration parameters
Azure OpenAI supports the same parameters as OpenAI:
#### Standard parameters
| Parameter | Range | Description |
| --------------------- | ---------- | ------------------------------------------------------------------ |
| **Temperature** | 0.0 - 2.0 | Controls randomness. Lower = more focused, higher = more creative. |
| **Max Output Tokens** | 1+ | Maximum length of the response |
| **Top P** | 0.0 - 1.0 | Nucleus sampling threshold. Alternative to temperature. |
| **Presence Penalty** | -2.0 - 2.0 | Penalize new topics (positive) or encourage them (negative) |
| **Frequency Penalty** | -2.0 - 2.0 | Penalize repetition (positive) or allow it (negative) |
| **Seed** | Integer | For reproducible outputs |
#### Advanced parameters
**Reasoning Effort:** Available on reasoning-optimized models (o-series and newer GPT models).
**Service Tier:** Available on newer models.
**Other parameters:**
* **JSON Mode:** Force valid JSON responses.
* **Parallel Tool Calls:** Execute multiple tools concurrently.
#### Azure-specific features
* **Deployment Management:** Models must be deployed before use.
* **Regional Availability:** Choose Azure regions for data residency.
* **Content Filtering:** Built-in content moderation and safety features.
* **Managed Identity:** Azure AD authentication support.
* **Private Endpoints:** VNet integration for secure access.
### Tool calling
* **Supported Tool Choices:** `auto`, `required`, `none`, or specific tool name.
* **Parallel Execution:** Yes.
## DeepSeek
Before you use this model, ensure you have a [DeepSeek API key](https://platform.deepseek.com/api_keys).
### Available models
DeepSeek offers general-purpose models, reasoning-optimized models (R-series), and coding-specialized models.
For the current list of available models, refer to [DeepSeek's documentation](https://platform.deepseek.com/api-docs/).
### Configuration parameters
| Parameter | Range | Description |
| --------------------- | ---------- | ----------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
| **Presence Penalty** | -2.0 - 2.0 | |
| **Frequency Penalty** | -2.0 - 2.0 | |
## Fireworks
Before you use this model, ensure you have a [Fireworks API key](https://fireworks.ai/api-keys).
### Available models
Fireworks provides high-speed inference for popular open-source models and fine-tuned variants, including:
* **Llama:** Meta's Llama models in various sizes.
* **Mixtral:** Mistral's mixture-of-experts models.
* **Qwen:** Alibaba's multilingual models.
* **DeepSeek:** DeepSeek models.
* **Other open models:** Gemma, Phi, and more.
For the current list of available models, refer to [Fireworks' model documentation](https://docs.fireworks.ai/models).
### Configuration parameters
| Parameter | Range | Description |
| --------------- | --------- | ----------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
### Tool calling
* **Supported Tool Choices:** `auto`, `required`, `none`.
* **Parallel Execution:** Yes.
## Google Gemini
Before you use this model, ensure you have a [Google AI API key](https://aistudio.google.com/app/apikey).
### Available models
Google offers Gemini models in multiple tiers (Ultra, Pro, Flash) optimized for different use cases.
For the current list of available models, refer to [Google's Gemini documentation](https://ai.google.dev/models/gemini).
### Configuration parameters
| Parameter | Range | Description |
| --------------------- | --------- | ----------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Output Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
| **Top K** | 1+ | Top-k sampling |
### Tool calling
* **Supported Tool Choices:** `auto`, `any`, `none`.
* **Parallel Execution:** No.
## Google Vertex AI
Before you use this model, ensure you have [Google Cloud credentials](https://cloud.google.com/vertex-ai/docs/authentication).
### Available models
Google offers Gemini models in multiple tiers (Ultra, Pro, Flash) optimized for different use cases, plus other models available through Vertex AI.
For the current list of available models, refer to the [Vertex AI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/models).
### Configuration parameters
| Parameter | Range | Description |
| --------------------- | --------- | ----------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Output Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
| **Top K** | 1+ | Top-k sampling |
#### Advanced options
* **Region Selection:** Deploy in specific Google Cloud regions.
* **Safety Settings:** Configure content filtering thresholds.
### Tool calling
* **Supported Tool Choices:** `auto`, `any`, `none`.
* **Parallel Execution:** No.
## Groq
Before you use this model, ensure you have a [Groq API key](https://console.groq.com/keys).
### Available models
Groq provides high-speed inference for popular open-source models including Llama, Mixtral, and Gemma variants.
For the current list of available models, refer to [Groq's model documentation](https://console.groq.com/docs/models).
### Configuration parameters
| Parameter | Range | Description |
| --------------- | --------- | ----------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
### Tool calling
* **Supported Tool Choices:** `auto`, `required`, `none`.
* **Parallel Execution:** Yes.
## Mistral AI
Before you use this model, ensure you have a [Mistral AI API key](https://console.mistral.ai/api-keys/).
### Available models
Mistral offers models in multiple tiers (Large, Medium, Small) optimized for different performance and cost requirements.
For the current list of available models, refer to [Mistral's documentation](https://docs.mistral.ai/platform/endpoints/).
### Configuration parameters
| Parameter | Range | Description |
| --------------- | --------- | ----------------------- |
| **Temperature** | 0.0 - 1.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
### Tool calling
* **Supported Tool Choices:** `auto`, `any`, `none`.
* **Parallel Execution:** No.
## OpenAI
Before you use this model, ensure you have an [OpenAI API key](https://platform.openai.com/api-keys) or [Azure OpenAI credentials](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart).
### Available models
OpenAI offers several model families with different capabilities and price points:
* **GPT series:** General-purpose chat models with various size/capability tiers.
* **o-series:** Reasoning-focused models optimized for complex problem-solving.
* **Legacy models:** Older GPT-3.5 and GPT-4 variants.
For the current list of available models, refer to the [OpenAI documentation](https://platform.openai.com/docs/models).
### Configuration parameters
Standard:
| Parameter | Range | Description |
| --------------------- | ---------- | ------------------------------------------------------------------ |
| **Temperature** | 0.0 - 2.0 | Controls randomness. Lower = more focused, higher = more creative. |
| **Max Output Tokens** | 1+ | Maximum length of the response |
| **Top P** | 0.0 - 1.0 | Nucleus sampling threshold. Alternative to temperature. |
| **Presence Penalty** | -2.0 - 2.0 | Penalize new topics (positive) or encourage them (negative) |
| **Frequency Penalty** | -2.0 - 2.0 | Penalize repetition (positive) or allow it (negative) |
| **Seed** | Integer | For reproducible outputs |
Advanced:
**Reasoning Effort**: Available on reasoning-optimized models (o-series and newer GPT models).
Controls reasoning depth before responding. Higher effort = better quality for complex tasks, longer latency.
| Value | Description |
| --------- | -------------------------------------------- |
| `none` | Disables reasoning (standard chat behavior) |
| `minimal` | Minimal reasoning |
| `low` | Light reasoning |
| `medium` | Moderate reasoning (default) |
| `high` | Deep reasoning |
| `xhigh` | Extra deep reasoning (if supported by model) |
When reasoning\_effort is active (not `none`), temperature, top\_p, and penalties are automatically disabled.
**Service Tier**: Available on newer models.
Controls request priority and processing allocation.
| Value | Description |
| ---------- | ---------------------------------------------------- |
| `auto` | System decides based on load (default) |
| `default` | Standard processing queue |
| `flex` | Lower cost, variable latency (if supported by model) |
| `priority` | High-priority queue, lower latency, higher cost |
**Other parameters:**
* **JSON Mode:** Force valid JSON responses.
* **Responses API:** Improved streaming (default: enabled).
* **Parallel Tool Calls:** Execute multiple tools concurrently.
### Tool calling
* **Supported Tool Choices:** `auto`, `required`, `none`, or specific tool name
* **Parallel Execution:** Yes
## OpenAI Compatible Endpoint
Authentication varies by endpoint (often API key or none).
### Configuration
**Required:**
* **Base URL:** Your endpoint URL (e.g., `https://your-endpoint.com/v1`).
* **Model Name:** Your model identifier.
Works with any framework or service that implements the OpenAI-compatible API format, including:
* Self-hosted open-source inference servers
* Model routing proxies
* Custom model endpoints
### Configuration parameters
All OpenAI-compatible parameters:
| Parameter | Range | Description |
| --------------------- | ---------- | ----------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
| **Frequency Penalty** | -2.0 - 2.0 | Reduce repetition |
| **Presence Penalty** | -2.0 - 2.0 | Encourage new topics |
**Advanced:**
* **JSON Mode:** If endpoint supports it.
* **Streaming:** If endpoint supports it.
* **Function Calling:** If endpoint implements OpenAI format.
### Tool calling
* **Supported Tool Choices:** `auto`, `required`, `none` (if endpoint supports).
* **Parallel Execution:** Yes (if endpoint supports).
### Example endpoints
**Local Ollama:**
```
Base URL: http://localhost:11434/v1
Model: llama3.1
```
**vLLM Server:**
```
Base URL: https://your-server.com/v1
Model: mistral-7b-instruct
```
**LiteLLM Proxy:**
```
Base URL: https://litellm.example.com
Model: gpt-4 (routes to configured backend)
```
## XAI
Before you use this model, ensure you have an [xAI API key](https://console.x.ai/).
### Available models
xAI offers Grok models in multiple sizes for different use cases.
For the current list of available models, refer to [xAI's documentation](https://docs.x.ai/docs).
### Configuration parameters
Standard OpenAI-compatible parameters:
| Parameter | Range | Description |
| --------------------- | --------- | -------------------------- |
| **Temperature** | 0.0 - 2.0 | Response randomness |
| **Max Tokens** | 1+ | Maximum response length |
| **Top P** | 0.0 - 1.0 | Nucleus sampling |
| **Presence Penalty** | 0 - 2.0 | Hidden on reasoning models |
| **Frequency Penalty** | 0 - 2.0 | Hidden on reasoning models |
### Tool calling
* **Supported Tool Choices:** OpenAI-compatible.
* **Parallel Execution:** Yes (if supported).
## Common Configuration Across All Providers
### Extra Parameters
All providers support a **JSON editor for extra parameters** not exposed in the UI:
```json theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
{
"logprobs": true,
"top_logprobs": 5,
"custom_parameter": "value"
}
```
**Use cases:**
* Provider-specific beta features
* Advanced parameters not yet in UI
* Custom metadata for tracking
**Limitation:** Cannot override parameters already in the UI (e.g., can't set temperature here if it's set above)
### Rate Limiting
**Requests Per Second (RPS)** - Available for all providers when running over datasets:
* **Range:** 0 - 500 RPS
* **Purpose:** Respect API rate limits, control costs
* **Default:** Varies by provider
Set this when running experiments or evaluations to avoid hitting rate limits.
## Next steps
Learn how to create and manage model configurations in the Playground.
Get started building prompts with your chosen model provider.
***
[Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
[Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/playground-model-providers.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
Amazon Bedrock
Anthropic
Azure OpenAI
DeepSeek
Fireworks
Google Gemini
Groq
Mistral AI
OpenAI
XAI