> ## Documentation Index
> Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Truefoundry integrations

> Integrate with Truefoundry using LangChain Python.

TrueFoundry provides an enterprise-ready [AI Gateway](https://www.truefoundry.com/ai-gateway) to provide governance and observability to agentic frameworks like LangChain. TrueFoundry AI Gateway serves as a unified interface for LLM access, providing:

* **Unified API Access**: Connect to 250+ LLMs (OpenAI, Claude, Gemini, Groq, Mistral) through one API
* **Low Latency**: Sub-3ms internal latency with intelligent routing and load balancing
* **Enterprise Security**: SOC 2, HIPAA, GDPR compliance with RBAC and audit logging
* **Quota and cost management**: Token-based quotas, rate limiting, and comprehensive usage tracking
* **Observability**: Full request/response logging, metrics, and traces with customizable retention

## Prerequisites

Before integrating LangChain with TrueFoundry, ensure you have:

1. **TrueFoundry Account**: A [TrueFoundry account](https://www.truefoundry.com/register) with at least one model provider configured. Follow the [TrueFoundry quick start guide](https://docs.truefoundry.com/gateway/quick-start)
2. **Personal Access Token**: Generate a token by following the [TrueFoundry token generation guide](https://docs.truefoundry.com/gateway/authentication)

## Quickstart

You can connect to TrueFoundry's unified LLM gateway through the [`ChatOpenAI`](https://reference.langchain.com/python/langchain-openai/chat_models/base/ChatOpenAI) interface.

* Set the `base_url` to your TrueFoundry endpoint (explained below)
* Set the `api_key` to your TrueFoundry [PAT (Personal Access Token)](https://docs.truefoundry.com/gateway/authentication#personal-access-token-pat)
* Use the same `model-name` as shown in the unified code snippet

### Installation

```bash pip theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
pip install langchain-openai
```

```bash uv theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
uv add langchain-openai
```

### Basic setup

Connect to TrueFoundry by updating the [`ChatOpenAI`](https://reference.langchain.com/python/langchain-openai/chat_models/base/ChatOpenAI) model in LangChain:

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    api_key=TRUEFOUNDRY_API_KEY,
    base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,
    model="openai-main/gpt-5.5"  # Similarly you can call any model from any model provider
)

llm.invoke("What is the meaning of life, universe and everything?")
```

The request is routed through your TrueFoundry gateway to the specified model provider. TrueFoundry automatically handles rate limiting, load balancing, and observability.

### LangGraph integration

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState
from langchain.messages import HumanMessage

# Define your LangGraph workflow
def call_model(state: MessagesState):
    model = ChatOpenAI(
        api_key=TRUEFOUNDRY_API_KEY,
        base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,
        # Copy the exact model name from gateway
        model="openai-main/gpt-5.5"
    )
    response = model.invoke(state["messages"])
    return {"messages": [response]}

# Build workflow
workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.set_entry_point("agent")
workflow.set_finish_point("agent")

app = workflow.compile()

# Run agent through TrueFoundry
result = app.invoke({"messages": [HumanMessage(content="Hello!")]})
```

## Observability and governance

With the Metrics Dashboard, you can monitor and analyze:

* **Performance Metrics**: Track key latency metrics like Request Latency, Time to First Token (TTFS), and Inter-Token Latency (ITL) with P99, P90, and P50 percentiles
* **Cost and Token Usage**: Gain visibility into your application's costs with detailed breakdowns of input/output tokens and the associated expenses for each model
* **Usage Patterns**: Understand how your application is being used with detailed analytics on user activity, model distribution, and team-based usage
* **Rate Limiting & Load Balancing**: Configure limits, distribute traffic across models, and set up fallbacks

## Support

For questions, issues, or support:

* **Email**: [support@truefoundry.com](mailto:support@truefoundry.com)
* **Documentation**: [https://docs.truefoundry.com/](https://docs.truefoundry.com/)

***

[Connect these docs](https://docs.langchain.com/use-these-docs.md) to your agent of choice via MCP for real-time answers.

[Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/providers/truefoundry.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).