> ## Documentation Index
> Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Trace LiteLLM applications

[LiteLLM](https://www.litellm.ai/) provides a unified interface for calling LLM providers using a consistent OpenAI-compatible API. It can be used either as a [Python SDK](https://docs.litellm.ai/docs/#litellm-python-sdk) embedded directly in your application, or as a [proxy server](https://docs.litellm.ai/docs/simple_proxy) that exposes an OpenAI-compatible endpoint for client applications.

This guide shows you how to trace LiteLLM calls with LangSmith using:

* The [LangSmith SDK](#use-langsmith_tracing-and-traceable) (`@traceable`) for application-level tracing.
* [LiteLLM’s built-in langsmith callback](#log-litellm-call-with-the-langsmith-callback) for model-level logging.
* The [LiteLLM Proxy](#use-the-litellm-proxy) for gateway-level tracing.

## Installation

Install the following when using either the LiteLLM Python SDK or LiteLLM Proxy:

<CodeGroup>
  ```bash Python SDK theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
  pip install litellm langsmith openai
  ```

  ```bash Proxy usage theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
  npm install openai langsmith
  ```
</CodeGroup>

The examples in this guide use OpenAI models, but you can install the necessary provider for your use case.

## Use the LiteLLM Python SDK

LiteLLM supports two ways to send traces to LangSmith, which operate at different layers:

* [LangSmith SDK tracing](#use-langsmith_tracing-and-traceable) with `LANGSMITH_TRACING=true` enables application-level tracing via the LangSmith SDK. This is useful when you want to trace broader business logic, multi-step pipelines, or spans created with `@traceable`.
* LiteLLM’s built-in [`langsmith` callback](#log-litellm-call-with-the-langsmith-callback) logs model calls directly from LiteLLM. This is recommended when you want to trace LiteLLM requests specifically, or run async applications.

<Note>
  Avoid enabling LiteLLM's `langsmith` callback and LangSmith tracing for the same LiteLLM calls, as this can result in duplicate traces.
</Note>

### Use `LANGSMITH_TRACING` and `traceable`

You can use `LANGSMITH_TRACING=true` together with `@traceable` for predictable traces in LangSmith. This approach ensures that the **Input** and **Output** columns reflect your function arguments and return values, allowing you to preserve the full message structure (including `role` and `content`). It also works reliably in simple synchronous scripts, without requiring an asyncio event loop or additional callback configuration.

1. Set the following environment variables to enable LangSmith tracing for LiteLLM Python SDK usage:

   ```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   export LANGSMITH_API_KEY="your_api_key"
   export LANGSMITH_PROJECT="litellm-integration"
   export LANGSMITH_TRACING="true"
   ```

   Create LangSmith [API keys](/langsmith/create-account-api-key) in the [LangSmith UI](https://smith.langchain.com?utm_source=docs\&utm_medium=cta\&utm_campaign=langsmith-signup\&utm_content=langsmith-trace-litellm).

   Depending on what provider you're using, you'll also need to set API keys:

   ```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   export OPENAI_API_KEY="your_openai_key"
   ```

2. Add the following code to your script file:

   ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   from langsmith import traceable
   from litellm import completion

   @traceable(name="LiteLLM Chat Completion")
   def run(messages):
       response = completion(
           model="gpt-4o",
           messages=messages,
       )
       # Return the assistant message so the LangSmith UI shows role + content
       return response["choices"][0]["message"]

   messages = [
       {"role": "user", "content": "Explain observability in LLM systems."}
   ]

   result = run(messages)
   print(result["content"])
   ```

   `@traceable` instruments your function as a LangSmith run. When `LANGSMITH_TRACING=true` is set, LangSmith automatically:

   * Creates a run when the function is invoked.
   * Records the function arguments as the run inputs.
   * Executes the function body (including the LiteLLM call).
   * Records the function's return value as the run output.
   * Captures timing, errors, and nested spans (if any).

   In this example, the `messages` argument becomes the trace input, and the returned assistant message object becomes the trace output. The LiteLLM call itself runs normally—`@traceable` wraps it with observability rather than modifying its behavior. This approach traces your application logic, not just the model call.

   <Tip>
     For more general examples using `@traceable`, refer to the [Custom instrumentation](/langsmith/annotate-code#use-@traceable-/-traceable) page.
   </Tip>

### Log LiteLLM call with the `langsmith` callback

LiteLLM can send traces directly to LangSmith using its built-in [callback system](https://docs.litellm.ai/docs/observability/callbacks). This is useful when running LiteLLM inside an async Python service and you want LiteLLM itself to emit model-level logs.

LiteLLM callbacks run in an asynchronous environment. When making asynchronous calls with `litellm.acompletion()`, you can enable the `langsmith` callback to log successful model calls.

<Tip>
  This approach is best suited for async applications. For simple synchronous scripts, use the `@traceable` method shown in the [previous section](#use-langsmith_tracing-and-traceable).
</Tip>

1. Set the following environment variables:

   ```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   export LANGSMITH_API_KEY="your_api_key"
   export LANGSMITH_PROJECT="litellm-integration"
   ```

   Create LangSmith [API keys](/langsmith/create-account-api-key) in the [LangSmith UI](https://smith.langchain.com?utm_source=docs\&utm_medium=cta\&utm_campaign=langsmith-signup\&utm_content=langsmith-trace-litellm).

   Depending on what provider you're using, you'll also need to set API keys:

   ```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   export OPENAI_API_KEY="your_openai_key"
   ```

2. To run this in a minimal script:

   * Use `acompletion()` (async API).
   * Run with `asyncio.run(...)` to create an event loop.
   * Set `langsmith_batch_size = 1` to flush immediately.

   ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   import asyncio
   import litellm
   from litellm import acompletion

   # Enable LiteLLM → LangSmith callback
   litellm.success_callback = ["langsmith"]

   # For short-lived scripts, send immediately instead of waiting for batch flush
   litellm.langsmith_batch_size = 1

   async def main():
       response = await acompletion(
           model="gpt-4o",
           messages=[
               {"role": "user", "content": "Explain observability in LLM systems."}
           ],
       )

       # Print the assistant message content for local verification
       print(response["choices"][0]["message"]["content"])

       # Allow time for background logger to flush before process exit
       await asyncio.sleep(1)

   if __name__ == "__main__":
       asyncio.run(main())
   ```

   The callback sends LiteLLM’s model request and response data directly to LangSmith, including provider metadata and token usage. Because LiteLLM controls the payload, the **Input** and **Output** columns may include additional metadata compared to the `@traceable` example.

## Use the LiteLLM Proxy

The LiteLLM proxy runs as a standalone server and exposes an OpenAI-compatible API.

1. To have the proxy log requests directly to LangSmith, configure the callback in `config.yaml`:

   ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   model_list:
     - model_name: gpt-4o
       litellm_params:
         model: openai/gpt-4o

   litellm_settings:
     callbacks: ["langsmith"]
   ```

2. Set environment variables in your proxy environment:

   ```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   export LANGSMITH_API_KEY="your_api_key"
   export LANGSMITH_PROJECT="litellm-proxy"
   export OPENAI_API_KEY="your_openai_key"
   ```

   <Note>
     The LiteLLM proxy runs as a separate service. If you enable LangSmith tracing at the proxy level, you must configure `LANGSMITH_API_KEY` and related environment variables in the proxy’s runtime environment. These settings are not shared with your application process.
   </Note>

3. Start the proxy:

   ```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
   litellm --config config.yaml
   ```

   By default, the proxy runs at `http://localhost:4000/v1`. Your application calls it using any OpenAI-compatible client (Python, JavaScript, curl, etc.).

   With `callbacks: ["langsmith"]` enabled, the proxy sends model request and response data directly to LangSmith. No tracing configuration is required in the client application.

4. Call the proxy from another terminal window:

   <CodeGroup>
     ```python Python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
     from openai import OpenAI

     client = OpenAI(
         base_url="http://localhost:4000/v1",
         api_key="anything"  # proxy may require a key but doesn't validate it by default
     )

     response = client.chat.completions.create(
         model="gpt-4o",
         messages=[
             {"role": "user", "content": "What is LiteLLM?"}
         ],
     )

     print(response.choices[0].message.content)
     ```

     ```javascript JavaScript theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
     import OpenAI from "openai";

     const client = new OpenAI({
     apiKey: "anything",
     baseURL: "http://localhost:4000/v1",
     });

     const response = await client.chat.completions.create({
     model: "gpt-4o",
     messages: [
         { role: "user", content: "Explain LiteLLM tracing." }
     ],
     });

     console.log(response.choices[0].message.content);
     ```
   </CodeGroup>

   The client sends a normal chat completion request, and the proxy handles provider routing and response formatting.

## Next steps

* [View traces in LangSmith](/langsmith/filter-traces-in-application)
* [Add custom metadata](/langsmith/ls-metadata-parameters)
* [Filter and sample traces](/langsmith/sample-traces)

***

<div className="source-links">
  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>

  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/trace-litellm.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>
</div>