Skip to main content
Smithtune is in beta.
Smithtune is a command-line tool for fine-tuning models on trajectories recorded in LangSmith. It trains a model with Fireworks or Baseten and compares the base and tuned models in a LangSmith experiment. You can optionally deploy the tuned model as an endpoint for your application.

​
How Smithtune works

1

Create a dataset

Select trajectories from a tracing project with a trace query filter, a model judge, or both.
2

Prepare data

Validate the trajectories, then split them into training, validation, and test data. Each trajectory stays in one split.
3

Train a model

Preview the run, then fine-tune a supported model with your provider.
4

Compare results

Evaluate the base and tuned models on the test data, then compare the experiments in LangSmith. Evaluation does not require a deployed endpoint.
5

Deploy an endpoint (optional)

Serve the tuned model for your application. To run an agent that uses it in production, see LangSmith Deployment.
To have your coding agent drive the fine-tuning process end to end, use the Smithtune skill.
Evaluation scores measure how closely each model matches the recorded behavior, not whether it completes tasks end to end. Smithtune does not execute the tool calls the model generates.

​
Set up Smithtune

Smithtune requires trajectories in a tracing project or a LangSmith trajectory dataset. You also need API keys for LangSmith, your training provider, and a judge model. Training, evaluation, and deployed endpoints incur charges from the services you use. To set up Smithtune:
  1. Install Smithtune and the LangSmith CLI, and set your API keys. Follow the Smithtune README for the install command and the environment variables each provider needs.
  2. Read the data rights and permitted use terms linked from the README, then acknowledge them before your first run:
  3. Check your local setup:
For the commands in each step, see the README or run smithtune --help.

​
See also

Connect these docs to your agent of choice via MCP for real-time answers.
Edit this page on GitHub or file an issue.