Skip to main content
The Gemini Decisions API reads text, structured JSON, and media, and returns typed answers and probabilities instead of generated text. GoogleClassifier exposes these decisions as a LangChain Runnable, so you can invoke, batch, or compose them with other runnables. It accepts strings, JSON objects and arrays, LangChain message objects, and standard media blocks, and records traces and token usage in LangSmith. You can also use it with agent middleware to route models and gate tool calls. Use it for focused decisions such as routing a request, flagging content for review, or rating severity, where application code acts on the probability or confidence.
GoogleClassifier is experimental and requires @langchain/google 0.2.16 or later and @langchain/core 1.4.2 or later. Its API may change without notice.

Setup

Create an API key in Google AI Studio and export it:
GoogleClassifier reads GOOGLE_API_KEY. Use the Gemini Developer API (Google AI Studio).

Quickstart

Create a GoogleClassifier with a decisions model, then pass the state and named questions together in each invocation. Questions that share state are answered in one request:
Answers are keyed by the question names you supplied, and grouped by type on nouls, choices, and scores. All answers are also available on answers. Answer types are inferred from the questions, so response.choices.department.choice is typed as "billing" | "shipping" | "other". The response carries the model that answered and token usage in usage_metadata. The question and response types, such as Noul and ClassifierResponse, come from @langchain/core/decisions. Every question requires instructions. Write the full question there, even when its name looks self-explanatory.

Question types

Each question is one of three types: Descriptions, criteria, and instructions can be text or structured JSON. JSON content is sent as JSON text. The Decisions API accepts only text in questions, so a media block in instructions throws InvalidInputError. @langchain/core/decisions also defines a box question type for locating objects in images. The Gemini Decisions API does not support it, and GoogleClassifier throws InvalidInputError for a request that includes one.

State

The state can be a string, a JSON object or array, LangChain messages, or standard media blocks:
  • Strings: Sent as-is.
  • JSON objects and arrays: Sent as JSON text. LangChain messages nested inside them are converted to text transcripts.
  • LangChain messages: Converted to text, one entry per message, each starting with its role (for example, user: or assistant:). Media blocks inside a message are sent as native media.
  • Media blocks: Standard image, audio, video, and file blocks with base64 data or a URL are sent as native media when they appear at the root of state, as an element of a root state array, or inside a root message. Media nested inside a JSON object throws InvalidInputError.
To classify a conversation, pass the messages as the state:
To classify an image, include a standard media block in a root state array. Replace delivery.jpg with the path to your image:

Configuration

Pass the model name as the first argument, and options as the second, or pass a single object with model: To authenticate with Application Default Credentials or googleAuthOptions instead of an API key, import GoogleClassifier from @langchain/google/node.

Errors

  • ConfigurationError: Thrown when the model name is missing or empty.
  • AuthError: Thrown when no apiKey or GOOGLE_API_KEY is available (and no credentials are configured for the Node entry point).
  • InvalidInputError: Thrown when a request includes a box question, when question instructions contain a media block, or when a media block is nested inside a JSON object in state.
  • PromptRefusedError: Thrown when the API blocks the input for safety. The message includes the block reason. Check for it with ModelRefusalError.isInstance(error) from @langchain/core/errors.

Agent middleware

The langchain package includes two beta middleware that use a decision model at an agent’s decision points. Pass a GoogleClassifier as their classifier. Both require langchain 1.6.5 or later:

Route models

modelRouterMiddleware picks one of your named models for each agent run, based on the latest human message, and stores the full choice answer in agent state as modelRoute:

Gate tool calls

autoModeMiddleware checks each call to the tools you list before it runs. It asks the classifier for the probability that the call is risky or insufficiently authorized, and blocks calls at or above 0.5:
With these criteria, deleting a file counts as risky, so the middleware returns an error ToolMessage instead of running the tool, and the agent reports that the call was blocked. Without criteria, the default instructions allow calls the user explicitly asked for, so the same request runs the tool. Tools not listed in tools run without classification.

Tracing

GoogleClassifier runs are traced in LangSmith with the google provider and model name, so you can inspect decisions and token usage alongside the rest of your application.

See also