The LLM Gateway is in beta.
Configure Azure Foundry
Azure Foundry supports either an API key or a Microsoft Entra ID service principal. Service principal authentication lets you use resources that disable local authentication. Both methods requireAZURE_FOUNDRY_RESOURCE_NAME. Set it to the Azure resource name, not a URL or deployment name. The gateway sends requests to <resource-name>.openai.azure.com in the Azure public cloud.
For API key authentication, add AZURE_FOUNDRY_API_KEY and AZURE_FOUNDRY_RESOURCE_NAME through the Azure Foundry provider form, as described in Admin setup.
Use an Entra ID service principal
To use an Entra ID service principal instead of an API key:-
Deploy a model on a Foundry resource, or use an existing deployment. Record the resource name and the deployment name. Use the deployment name as the
modelvalue in requests. - Register an application in Microsoft Entra ID, or use an existing app registration. Record its directory (tenant) ID and application (client) ID.
- Add a client secret to the app registration. Record the secret’s value, not its ID.
-
On the Foundry resource, not a project inside it, assign the app’s service principal the
Foundry Userrole (formerlyAzure AI User). The genericOwner,Contributor, andReaderroles do not grant permission to call models. -
In LangSmith, select the workspace that routes Azure requests. Go to Settings > Integrations > Provider Secrets and add each of the following as a custom secret:
Select Custom in the provider selector to enter each secret’s name and value. The Azure Foundry provider form only exposes API key authentication.
-
Remove
AZURE_FOUNDRY_API_KEYfrom the workspace if it is set. An API key takes precedence over service principal credentials, even when the resource disables API key authentication.
AZURE_CLIENT_SECRET when you rotate the secret in Entra ID.
Verify the setup
After you configure gateway access, send a test request through the gateway with your LangSmith API key:chat.completion response confirms the setup. For other Cloud regions, replace the hostname with your regional gateway.
If Azure returns PermissionDenied, confirm that the Foundry User role is assigned on the Foundry resource, not a project. A new role assignment can take up to 10 minutes to take effect.
See also
- Admin setup: add provider secrets and grant gateway access.
- Direct model access: call provider APIs through provider-specific gateway paths.
- Self-hosted LLM Gateway: enable the gateway on self-hosted LangSmith.
Connect these docs to your agent of choice via MCP for real-time answers.

