Skip to main content
The LLM Gateway is in beta.
The gateway authenticates to each model provider with credentials stored in your workspace’s Provider Secrets. Most providers need only an API key, which you add in Admin setup. Some providers also support identity-based authentication, which lets you use the gateway when your organization disables provider API keys.

Configure Azure Foundry

Azure Foundry supports either an API key or a Microsoft Entra ID service principal. Service principal authentication lets you use resources that disable local authentication. Both methods require AZURE_FOUNDRY_RESOURCE_NAME. Set it to the Azure resource name, not a URL or deployment name. The gateway sends requests to <resource-name>.openai.azure.com in the Azure public cloud. For API key authentication, add AZURE_FOUNDRY_API_KEY and AZURE_FOUNDRY_RESOURCE_NAME through the Azure Foundry provider form, as described in Admin setup.

Use an Entra ID service principal

To use an Entra ID service principal instead of an API key:
  1. Deploy a model on a Foundry resource, or use an existing deployment. Record the resource name and the deployment name. Use the deployment name as the model value in requests.
  2. Register an application in Microsoft Entra ID, or use an existing app registration. Record its directory (tenant) ID and application (client) ID.
  3. Add a client secret to the app registration. Record the secret’s value, not its ID.
  4. On the Foundry resource, not a project inside it, assign the app’s service principal the Foundry User role (formerly Azure AI User). The generic Owner, Contributor, and Reader roles do not grant permission to call models.
  5. In LangSmith, select the workspace that routes Azure requests. Go to Settings > Integrations > Provider Secrets and add each of the following as a custom secret: Select Custom in the provider selector to enter each secret’s name and value. The Azure Foundry provider form only exposes API key authentication.
  6. Remove AZURE_FOUNDRY_API_KEY from the workspace if it is set. An API key takes precedence over service principal credentials, even when the resource disables API key authentication.
All three Entra ID secrets are required. The gateway obtains and refreshes Azure access tokens automatically. Users continue to authenticate to the gateway with their LangSmith API key, not the Azure credentials. This method uses a client secret. It does not use managed identity or workload identity federation. Client secrets expire, so update AZURE_CLIENT_SECRET when you rotate the secret in Entra ID.

Verify the setup

After you configure gateway access, send a test request through the gateway with your LangSmith API key:
A chat.completion response confirms the setup. For other Cloud regions, replace the hostname with your regional gateway. If Azure returns PermissionDenied, confirm that the Foundry User role is assigned on the Foundry resource, not a project. A new role assignment can take up to 10 minutes to take effect.

See also