View evaluators
In the LangSmith UI, select Evaluators in the left sidebar to view all evaluators in your workspace. The evaluators table shows the following columns:| Column | Description |
|---|---|
| Name | The evaluator name |
| Type | LLM, Code, or Composite score. Composite score evaluators are scoped to individual tracing projects and datasets and do not appear here. |
| Feedback Key | The feedback key the evaluator produces |
| Resources | Tracing projects and datasets this evaluator is attached to |
| Created By | The workspace member who created the evaluator |
| Updated At | When the evaluator was last modified |
| Created At | When the evaluator was created |
Create an evaluator
- In the LangSmith UI, select Evaluators in the left sidebar.
- Click + Evaluator to open the Add Evaluator panel.
- Choose one of the following:
- Create from scratch: Build a new LLM-as-a-Judge, Code, or Composite evaluator.
- Attach an existing evaluator: Select an evaluator already in your workspace to reuse it across additional resources.
-
Create from a template: Start from a ready-made evaluator for common evaluation patterns. Templates are organized by the following categories:
Category Description Security Detect leaks, injections, and adversarial inputs Safety Evaluate content safety and moderation Quality Measure output quality and accuracy Conversation Evaluate conversational quality and user experience Trajectory Evaluate agent tool use and decision paths Image & Voice (Multi-Modal) Evaluate image content quality and safety, as well as voice and audio interaction quality
View evaluator details
Click any evaluator in the table to open its detail view. The detail view has four tabs:- Overview: The evaluator’s feedback configuration and prompt or code definition.
- Traces: Traces processed by this evaluator across all attached resources.
- Logs: Execution logs for this evaluator across all attached resources.
- Resources: The tracing projects and datasets this evaluator is attached to.
Edit an evaluator
Open an evaluator and update its configuration in the Overview tab. Because the evaluator is shared, changes apply across all tracing projects and datasets it is attached to.Delete an evaluator
An evaluator cannot be deleted while it is attached to a tracing project or dataset. To delete an evaluator, first remove it from all resources via the Resources tab, then delete it.Connect these docs to Claude, VSCode, and more via MCP for real-time answers.