OpenAI Decisions integration

The OpenAI Decisions API reads natural language and images, and returns typed answers and calibrated probabilities instead of generated text. OpenAIDecisions exposes these decisions as a LangChain Runnable, so you can invoke, batch, or compose them with other runnables. It accepts strings, structured JSON, and LangChain message objects, and records traces and token usage in LangSmith. Use it for focused decisions such as routing a request, choosing a model, or checking whether a tool call is safe to run. Use OpenAI decisions in LangChain middleware to control createAgent behavior at specific lifecycle points, including with custom middleware that classifies agent state. See Agent middleware for examples.

OpenAIDecisions and its middleware are in beta and require @langchain/openai>=1.8.0. The middleware also requires langchain>=1.6.0. Their APIs may change without notice.

Setup

npm install @langchain/openai @langchain/core

yarn add @langchain/openai @langchain/core

pnpm add @langchain/openai @langchain/core

Create an API key in the OpenAI dashboard and export it:

export OPENAI_API_KEY=...

Optional: Set OPENAI_BASE_URL to use a compatible gateway or test server. The default is the OpenAI API endpoint.

export OPENAI_BASE_URL=https://gateway.example.com/v1

To route requests through the LangSmith LLM gateway, set LANGSMITH_GATEWAY=true. Gateway requests authenticate with LANGSMITH_GATEWAY_API_KEY or LANGSMITH_API_KEY:

export LANGSMITH_GATEWAY=true
export LANGSMITH_API_KEY=...

Quickstart

Create an OpenAIDecisions instance with a decisions model, then pass the input and named questions together in each invocation. Questions that share input are answered in one request:

import { OpenAIDecisions } from "@langchain/openai";

const decisions = new OpenAIDecisions("gpt-6-luna");

const response = await decisions.invoke({
  input:
    "The deploy failed twice and customers are seeing 500s. Can someone look now?",
  questions: {
    urgent: {
      type: "predicate",
      instructions: "Does this need attention right now?",
    },
    team: {
      type: "choice",
      instructions: "Which team should pick this up?",
      choices: {
        infra: "Deploys, availability, and on-call incidents.",
        billing: "Payments, invoices, and subscriptions.",
      },
    },
    severity: {
      type: "score",
      instructions: "How severe is the impact?",
      levels: [
        "Cosmetic",
        { label: "Degraded", description: "Degraded for some users." },
        { label: "Outage", description: "Full outage." },
      ],
    },
  },
});

const { urgent, team, severity } = response.answers;
if (urgent.type === "predicate") console.log(urgent.probability);
if (team.type === "choice") console.log(team.choice, team.confidence);
if (severity.type === "score") console.log(severity.score);

Answers are keyed by the question names you supplied, and their types are inferred from the questions: team.choice is typed as "infra" | "billing". Every question receives either an answer of its own type or a refusal, so check type before reading a typed field. The response also carries the model that answered, token usage in usage_metadata, and the OpenAI requestId. The request input can be a string, a JSON object or array, or LangChain messages. Messages, including system, AI, and tool messages, are sent natively as a conversation, with tool calls and tool results as function call items. Text and image content is kept. Other content, such as files or audio, is replaced by a placeholder or omitted. JSON objects and arrays are sent as structured JSON in a single user message, and any messages nested inside them are converted the same way.

Decision types

Each question is one of three types:

Type	Asks	Returns	Reach for it when
`predicate`	Is this true?	`probability`, the probability of yes	Your code branches on an `if`
`choice`	Which of these options?	`choice`, plus `probabilities` and `confidence`	Options map to distinct code paths
`score`	Which level?	`score`, plus `probabilities` and `confidence`	The answer is a spectrum you compare to a threshold

predicate is the only one without a confidence, since the probability is the answer. Use score rather than a predicate for a spectrum: a probability of 0.5 means an even split between yes and no, not “medium”. score levels are ordered from lowest to highest and indexed from 0. Pass a plain string for a label-only level, or an object with a label and description. The returned score is the probability-weighted average of the level indices, so it may fall between levels. Each entry in probabilities holds a level’s index as value, its label, and its probability. When the model declines to answer a question, the answer is a RefusalAnswer with type: "refusal" instead of a typed answer.

Agent middleware

The langchain package includes two beta middleware that put OpenAI decisions on an agent’s decision points. Import them from langchain. Install langchain alongside the OpenAI integration before using them:

npm install langchain @langchain/openai @langchain/core

yarn add langchain @langchain/openai @langchain/core

pnpm add langchain @langchain/openai @langchain/core

Middleware	Hook	Decision
`openAIModelRouterMiddleware`	`beforeAgent`, `wrapModelCall`	Which model handles the run
`openAIAutoModeMiddleware`	`wrapToolCall`	Whether a tool call is too risky to run

Both middleware require a model: a configured OpenAIDecisions instance.

Model routing

Classifies the latest human message across your named models and uses the selected one for every model call in the run. Each choice pairs a model with the criterion for picking it:

import { createAgent, openAIModelRouterMiddleware } from "langchain";
import { OpenAIDecisions } from "@langchain/openai";

const router = openAIModelRouterMiddleware({
  choices: {
    fast: {
      model: "openai:gpt-6-luna",
      criteria:
        "Direct lookups, extraction, and localized changes with explicit targets.",
    },
    powerful: {
      model: "openai:gpt-6-sol",
      criteria:
        "Architecture, novel root-cause reasoning, and high-stakes decisions.",
    },
  },
  instructions: "Choose the least costly model that can complete the task safely.",
  model: new OpenAIDecisions("gpt-6-luna"),
});

const agent = createAgent({ model: "openai:gpt-6-luna", middleware: [router] });

const result = await agent.invoke({
  messages: [
    {
      role: "user",
      content:
        "Design a Byzantine fault tolerant consensus protocol and prove its safety and liveness properties.",
    },
  ],
});
console.log(result.modelRoute?.choice);

The answer is stored in agent state under modelRoute as an object with choice, probabilities, and confidence. When the model refuses, or the state has no human message, modelRoute is null and model calls use the agent’s own model.

Tool-risk gating

Asks for the probability that a tool call is risky or insufficiently authorized. Calls with a probability of 0.5 or higher return an error ToolMessage instead of running the tool. Only the tools you list are classified. Pass their names or tool objects:

import { createAgent, openAIAutoModeMiddleware, tool } from "langchain";
import { OpenAIDecisions } from "@langchain/openai";
import * as z from "zod";

const deleteAllBackups = tool(async () => "Backups deleted.", {
  name: "delete_all_backups",
  description: "Delete every backup. This action cannot be undone.",
  schema: z.object({}),
});

const agent = createAgent({
  model: "openai:gpt-6-sol",
  tools: [deleteAllBackups],
  middleware: [
    openAIAutoModeMiddleware({
      tools: [deleteAllBackups],
      model: new OpenAIDecisions("gpt-6-luna"),
    }),
  ],
});

const result = await agent.invoke({
  messages: [{ role: "user", content: "Delete all backups." }],
});
console.log(result.messages.at(-1)?.text);

Override instructions to describe risk for your own tools. The middleware fails closed: refusals block the call, and API errors propagate without running the tool.

Do not put secrets in tool arguments or conversation state unless sending them to OpenAI is acceptable. This middleware refuses risky calls. It does not request approval. Pair it with human-in-the-loop middleware when you want a person to approve them.

Custom middleware

OpenAIDecisions accepts LangChain message objects directly, so a custom middleware hook can classify an agent’s conversation state without converting it first. This example classifies the conversation once at the start of each run and stores the complete answer in agent state:

import { createAgent, createMiddleware } from "langchain";
import { OpenAIDecisions } from "@langchain/openai";
import * as z from "zod";

const decisions = new OpenAIDecisions("gpt-6-luna");

const triageMiddleware = createMiddleware({
  name: "TriageMiddleware",
  stateSchema: z.object({
    triage: z
      .object({ choice: z.string(), confidence: z.number() })
      .optional(),
  }),
  beforeAgent: async (state) => {
    const { answers } = await decisions.invoke({
      input: state.messages,
      questions: {
        triage: {
          type: "choice",
          instructions: "Which team should handle this conversation?",
          choices: {
            billing: "Payments, invoices, and subscriptions.",
            infra: "Deploys, availability, and incidents.",
            other: "Requests that belong to another team.",
          },
        },
      },
    });
    if (answers.triage.type !== "choice") return;
    const { choice, confidence } = answers.triage;
    return { triage: { choice, confidence } };
  },
});

const agent = createAgent({
  model: "openai:gpt-6-luna",
  middleware: [triageMiddleware],
});
const result = await agent.invoke({
  messages: [{ role: "user", content: "Customers are seeing 500 errors." }],
});
console.log(result.triage?.choice, result.triage?.confidence);

Use any lifecycle hook that matches your decision point. For example, use beforeModel to reclassify after each tool result, or wrapToolCall to classify a proposed action. See Custom middleware for all hooks and state patterns.

Tracing

OpenAI decisions are traced in LangSmith, so you can inspect decisions and token usage alongside the rest of your agent.

Popular Providers

General integrations

RAG integrations

Setup

Quickstart

Decision types

Agent middleware

Model routing

Tool-risk gating

Custom middleware

Tracing

See also

​Setup

​Quickstart

​Decision types

​Agent middleware

​Model routing

​Tool-risk gating

​Custom middleware

​Tracing

​See also

Setup

Quickstart

Decision types

Agent middleware

Model routing

Tool-risk gating

Custom middleware

Tracing

See also