Observability in Studio

• Iterate on prompts: Modify prompts inside graph nodes directly or with the LangSmith playground.
• Run experiments over a dataset: Execute your assistant over a LangSmith dataset to score and compare results.
• Debug LangSmith traces: Import traced runs into Studio and optionally clone them into your local agent.
• Add a node to a dataset: Turn parts of thread history into dataset examples for evaluation or further analysis.

​

Iterate on prompts

• Direct node editing
• Playground interface

​

Direct node editing

​

Graph Configuration

langgraph_nodes

• Description: Specifies which nodes of the graph a configuration field is associated with.
• Value Type: Array of strings, where each string is the name of a node in your graph.
• Usage Context: Include in the json_schema_extra dictionary for Pydantic models or the metadata["json_schema_extra"] dictionary for dataclasses.
• Example:
  Copy

  system_prompt: str = Field(
      default="You are a helpful AI assistant.",
      json_schema_extra={"langgraph_nodes": ["call_model", "other_node"]},
  )
  

langgraph_type

• Description: Specifies the type of configuration field, which determines how it’s handled in the UI.
• Value Type: String
• Supported Values:
  • "prompt": Indicates the field contains prompt text that should be treated specially in the UI.
• Usage Context: Include in the json_schema_extra dictionary for Pydantic models or the metadata["json_schema_extra"] dictionary for dataclasses.
• Example:
  Copy

  system_prompt: str = Field(
      default="You are a helpful AI assistant.",
      json_schema_extra={
          "langgraph_nodes": ["call_model"],
          "langgraph_type": "prompt",
      },
  )
  

## Using Pydantic
from pydantic import BaseModel, Field
from typing import Annotated, Literal

class Configuration(BaseModel):
    """The configuration for the agent."""

    system_prompt: str = Field(
        default="You are a helpful AI assistant.",
        description="The system prompt to use for the agent's interactions. "
        "This prompt sets the context and behavior for the agent.",
        json_schema_extra={
            "langgraph_nodes": ["call_model"],
            "langgraph_type": "prompt",
        },
    )

    model: Annotated[
        Literal[
            "anthropic/claude-3-7-sonnet-latest",
            "anthropic/claude-3-5-haiku-latest",
            "openai/o1",
            "openai/gpt-4o-mini",
            "openai/o1-mini",
            "openai/o3-mini",
        ],
        {"__template_metadata__": {"kind": "llm"}},
    ] = Field(
        default="openai/gpt-4o-mini",
        description="The name of the language model to use for the agent's main interactions. "
        "Should be in the form: provider/model-name.",
        json_schema_extra={"langgraph_nodes": ["call_model"]},
    )

## Using Dataclasses
from dataclasses import dataclass, field

@dataclass(kw_only=True)
class Configuration:
    """The configuration for the agent."""

    system_prompt: str = field(
        default="You are a helpful AI assistant.",
        metadata={
            "description": "The system prompt to use for the agent's interactions. "
            "This prompt sets the context and behavior for the agent.",
            "json_schema_extra": {"langgraph_nodes": ["call_model"]},
        },
    )

    model: Annotated[str, {"__template_metadata__": {"kind": "llm"}}] = field(
        default="anthropic/claude-3-5-sonnet-20240620",
        metadata={
            "description": "The name of the language model to use for the agent's main interactions. "
            "Should be in the form: provider/model-name.",
            "json_schema_extra": {"langgraph_nodes": ["call_model"]},
        },
    )

​

Editing prompts in the UI

1. Locate the gear icon on nodes with associated configuration fields.
2. Click to open the configuration modal.
3. Edit the values.
4. Save to update the current assistant version or create a new one.

​

Playground

1. Select a thread.
2. Click View LLM Runs on a node. This lists all the LLM calls (if any) made inside the node.
3. Select an LLM run to open in the playground.
4. Modify prompts and test different model and tool settings.
5. Copy updated prompts back to your graph.

​

Run experiments over a dataset

​

Prerequisites

• A LangSmith dataset: Your dataset should contain the inputs you want to test and optionally, reference outputs for comparison. The schema for the inputs must match the required input schema for the assistant. For more information on schemas, see here. For more on creating datasets, refer to How to Manage Datasets.
• (Optional) Evaluators: You can attach evaluators (e.g., LLM-as-a-Judge, heuristics, or custom functions) to your dataset in LangSmith. These will run automatically after the graph has processed all inputs.
• A running application: The experiment can be run against:
  • An application deployed on LangGraph Platform.
  • A locally running application started via the langgraph-cli.

​

Experiment setup

1. Launch the experiment. Click the Run experiment button in the top right corner of the Studio page.
2. Select your dataset. In the modal that appears, select the dataset (or a specific dataset split) to use for the experiment and click Start.
3. Monitor the progress. All of the inputs in the dataset will now be run against the active assistant. Monitor the experiment’s progress via the badge in the top right corner.
4. You can continue to work in Studio while the experiment runs in the background. Click the arrow icon button at any time to navigate to LangSmith and view the detailed experiment results.

​

Debug LangSmith traces

​

Open deployed threads

1. Open the LangSmith trace, selecting the root run.
2. Click Run in Studio.

​

Testing local agents with remote traces

​

Prerequisites

• A LangSmith traced thread
• A locally running agent.

​

Clone thread

1. Open the LangSmith trace, selecting the root run.
2. Click the dropdown next to Run in Studio.
3. Enter your local agent’s URL.
4. Select Clone thread locally.
5. If multiple graphs exist, select the target graph.

​

Add node to dataset

1. Select a thread.
2. Click Add to Dataset.
3. Select nodes whose input/output you want to add to a dataset.
4. For each selected node, select the target dataset to create the example in. By default a dataset for the specific assistant and node will be selected. If this dataset does not yet exist, it will be created.
5. Edit the example’s input/output as needed before adding it to the dataset.
6. Select Add to dataset at the bottom of the page to add all selected nodes to their respective datasets.