Trace LiteLLM applications

• The LangSmith SDK (@traceable) for application-level tracing.
• LiteLLM’s built-in langsmith callback for model-level logging.
• The LiteLLM Proxy for gateway-level tracing.

​

Installation

pip install litellm langsmith openai

​

Use the LiteLLM Python SDK

• LangSmith SDK tracing with LANGSMITH_TRACING=true enables application-level tracing via the LangSmith SDK. This is useful when you want to trace broader business logic, multi-step pipelines, or spans created with @traceable.
• LiteLLM’s built-in langsmith callback logs model calls directly from LiteLLM. This is recommended when you want to trace LiteLLM requests specifically, or run async applications.

​

Use LANGSMITH_TRACING and traceable

1. Set the following environment variables to enable LangSmith tracing for LiteLLM Python SDK usage:
   export LANGSMITH_API_KEY="your_api_key"
   export LANGSMITH_PROJECT="litellm-integration"
   export LANGSMITH_TRACING="true"
   
   Create LangSmith API keys in the LangSmith UI. Depending on what provider you’re using, you’ll also need to set API keys:
   export OPENAI_API_KEY="your_openai_key"
   
2. Add the following code to your script file:
   from langsmith import traceable
   from litellm import completion
   
   @traceable(name="LiteLLM Chat Completion")
   def run(messages):
       response = completion(
           model="gpt-4o",
           messages=messages,
       )
       # Return the assistant message so the LangSmith UI shows role + content
       return response["choices"][0]["message"]
   
   messages = [
       {"role": "user", "content": "Explain observability in LLM systems."}
   ]
   
   result = run(messages)
   print(result["content"])
   
   @traceable instruments your function as a LangSmith run. When LANGSMITH_TRACING=true is set, LangSmith automatically:
   • Creates a run when the function is invoked.
   • Records the function arguments as the run inputs.
   • Executes the function body (including the LiteLLM call).
   • Records the function’s return value as the run output.
   • Captures timing, errors, and nested spans (if any).
   In this example, the messages argument becomes the trace input, and the returned assistant message object becomes the trace output. The LiteLLM call itself runs normally—@traceable wraps it with observability rather than modifying its behavior. This approach traces your application logic, not just the model call.
   For more general examples using @traceable, refer to the Custom instrumentation page.

​

Log LiteLLM call with the langsmith callback

1. Set the following environment variables:
   export LANGSMITH_API_KEY="your_api_key"
   export LANGSMITH_PROJECT="litellm-integration"
   
   Create LangSmith API keys in the LangSmith UI. Depending on what provider you’re using, you’ll also need to set API keys:
   export OPENAI_API_KEY="your_openai_key"
   
2. To run this in a minimal script:
   • Use acompletion() (async API).
   • Run with asyncio.run(...) to create an event loop.
   • Set langsmith_batch_size = 1 to flush immediately.
   import asyncio
   import litellm
   from litellm import acompletion
   
   # Enable LiteLLM → LangSmith callback
   litellm.success_callback = ["langsmith"]
   
   # For short-lived scripts, send immediately instead of waiting for batch flush
   litellm.langsmith_batch_size = 1
   
   async def main():
       response = await acompletion(
           model="gpt-4o",
           messages=[
               {"role": "user", "content": "Explain observability in LLM systems."}
           ],
       )
   
       # Print the assistant message content for local verification
       print(response["choices"][0]["message"]["content"])
   
       # Allow time for background logger to flush before process exit
       await asyncio.sleep(1)
   
   if __name__ == "__main__":
       asyncio.run(main())
   
   The callback sends LiteLLM’s model request and response data directly to LangSmith, including provider metadata and token usage. Because LiteLLM controls the payload, the Input and Output columns may include additional metadata compared to the @traceable example.

​

Use the LiteLLM Proxy

1. To have the proxy log requests directly to LangSmith, configure the callback in config.yaml:
   model_list:
     - model_name: gpt-4o
       litellm_params:
         model: openai/gpt-4o
   
   litellm_settings:
     callbacks: ["langsmith"]
   
2. Set environment variables in your proxy environment:
   export LANGSMITH_API_KEY="your_api_key"
   export LANGSMITH_PROJECT="litellm-proxy"
   export OPENAI_API_KEY="your_openai_key"
   
   The LiteLLM proxy runs as a separate service. If you enable LangSmith tracing at the proxy level, you must configure LANGSMITH_API_KEY and related environment variables in the proxy’s runtime environment. These settings are not shared with your application process.
3. Start the proxy:
   litellm --config config.yaml
   
   By default, the proxy runs at http://localhost:4000/v1. Your application calls it using any OpenAI-compatible client (Python, JavaScript, curl, etc.). With callbacks: ["langsmith"] enabled, the proxy sends model request and response data directly to LangSmith. No tracing configuration is required in the client application.
4. Call the proxy from another terminal window:
   Python

   JavaScript

   from openai import OpenAI
   
   client = OpenAI(
       base_url="http://localhost:4000/v1",
       api_key="anything"  # proxy may require a key but doesn't validate it by default
   )
   
   response = client.chat.completions.create(
       model="gpt-4o",
       messages=[
           {"role": "user", "content": "What is LiteLLM?"}
       ],
   )
   
   print(response.choices[0].message.content)
   
   The client sends a normal chat completion request, and the proxy handles provider routing and response formatting.

​

Next steps

• View traces in LangSmith
• Add custom metadata
• Filter and sample traces