IBM watsonx.ai integration

This will help you getting started with IBM watsonx.ai chat models. For detailed documentation of all IBM watsonx.ai features and configurations head to the IBM watsonx.ai.

Overview

Integration details

Class	Package	Serializable	PY support	Downloads	Version
`ChatWatsonx`	`@langchain/ibm`	✅	✅

Model features

Tool calling	Structured output	Image input	Audio input	Video input	Token-level streaming	Token usage	Logprobs
✅	✅	✅	❌	❌	✅	✅	❌

Setup

To access IBM watsonx.ai models you’ll need to create an IBM watsonx.ai account, get an API key, and install the @langchain/ibm integration package.

Credentials

Head to IBM Cloud to sign up to IBM watsonx.ai and generate an API key or provide any other authentication form as presented below.

IAM authentication

export WATSONX_AI_AUTH_TYPE=iam
export WATSONX_AI_APIKEY=<YOUR-APIKEY>

Bearer token authentication

export WATSONX_AI_AUTH_TYPE=bearertoken
export WATSONX_AI_BEARER_TOKEN=<YOUR-BEARER-TOKEN>

IBM watsonx.ai software authentication

export WATSONX_AI_AUTH_TYPE=cp4d
export WATSONX_AI_USERNAME=<YOUR_USERNAME>
export WATSONX_AI_PASSWORD=<YOUR_PASSWORD>
export WATSONX_AI_URL=<URL>

Once these are places in your environmental variables and object is initialized authentication will proceed automatically. Authentication can also be accomplished by passing these values as parameters to a new instance.

IAM authentication

import { ChatWatsonx } from "@langchain/ibm";

const props = {
  version: "YYYY-MM-DD",
  serviceUrl: "<SERVICE_URL>",
  projectId: "<PROJECT_ID>",
  watsonxAIAuthType: "iam",
  watsonxAIApikey: "<YOUR-APIKEY>",
};
const instance = new ChatWatsonx(props);

Bearer token authentication

import { ChatWatsonx } from "@langchain/ibm";

const props = {
  version: "YYYY-MM-DD",
  serviceUrl: "<SERVICE_URL>",
  projectId: "<PROJECT_ID>",
  watsonxAIAuthType: "bearertoken",
  watsonxAIBearerToken: "<YOUR-BEARERTOKEN>",
};
const instance = new ChatWatsonx(props);

IBM watsonx.ai software authentication

import { ChatWatsonx } from "@langchain/ibm";

const props = {
  version: "YYYY-MM-DD",
  serviceUrl: "<SERVICE_URL>",
  projectId: "<PROJECT_ID>",
  watsonxAIAuthType: "cp4d",
  watsonxAIUsername: "<YOUR-USERNAME>",
  watsonxAIPassword: "<YOUR-PASSWORD>",
  watsonxAIUrl: "<url>",
};
const instance = new ChatWatsonx(props);

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

Installation

The LangChain IBM watsonx.ai integration lives in the @langchain/ibm package:

npm install @langchain/ibm @langchain/core

yarn add @langchain/ibm @langchain/core

pnpm add @langchain/ibm @langchain/core

Instantiation

Now we can instantiate our model object and generate chat completions:

import { ChatWatsonx } from "@langchain/ibm";
const props = {
  maxTokens: 200,
  temperature: 0.5
};

const instance = new ChatWatsonx({
  version: "YYYY-MM-DD",
  serviceUrl: process.env.API_URL,
  projectId: "<PROJECT_ID>",
  // spaceId: "<SPACE_ID>",
  // idOrName: "<DEPLOYMENT_ID>",
  model: "<MODEL_ID>",
  ...props
});

Note:

You must provide spaceId, projectId or idOrName(deployment id) unless you use lightweight engine which works without specifying either (refer to watsonx.ai docs)
Depending on the region of your provisioned service instance, use correct serviceUrl.

Using model Gateway

import { ChatWatsonx } from "@langchain/ibm";
const props = {
  maxTokens: 200,
  temperature: 0.5
};

const instance = new ChatWatsonx({
  version: "YYYY-MM-DD",
  serviceUrl: process.env.API_URL,
  model: "<ALIAS_MODEL_ID>",
  modelGateway: true,
  ...props
});

To use model gateway with Langchain, you need to previously create a provider and add model via @ibm-cloud/watsonx-ai SDK or watsonx.ai API. Follow this documentation:

API.
SDK.

Invocation

const aiMsg = await instance.invoke([{
  role: "system",
  content: "You are a helpful assistant that translates English to French. Translate the user sentence.",
},
{
  role: "user",
  content: "I love programming."
}]);
console.log(aiMsg)

AIMessage {
  "id": "chat-c5341b2062dc42f091e5ae2558e905e3",
  "content": " J'adore la programmation.",
  "additional_kwargs": {
    "tool_calls": []
  },
  "response_metadata": {
    "tokenUsage": {
      "completion_tokens": 10,
      "prompt_tokens": 28,
      "total_tokens": 38
    },
    "finish_reason": "stop"
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 28,
    "output_tokens": 10,
    "total_tokens": 38
  }
}

console.log(aiMsg.content)

 J'adore la programmation.

Streaming the model output

import { HumanMessage, SystemMessage } from "@langchain/core/messages";

const messages = [
    new SystemMessage('You are a helpful assistant which telling short-info about provided topic.'),
    new HumanMessage("moon")
]
const stream = await instance.stream(messages);
for await(const chunk of stream){
    console.log(chunk)
}

 The
 Moon
 is
 Earth
'
s
 only
 natural
 satellite
 and

Tool calling

import { tool } from "@langchain/core/tools";
import * as z from "zod";

const calculatorSchema = z.object({
    operation: z
      .enum(["add", "subtract", "multiply", "divide"])
      .describe("The type of operation to execute."),
    number1: z.number().describe("The first number to operate on."),
    number2: z.number().describe("The second number to operate on."),
  });

const calculatorTool = tool(
async ({ operation, number1, number2 }) => {
    if (operation === "add") {
    return `${number1 + number2}`;
    } else if (operation === "subtract") {
    return `${number1 - number2}`;
    } else if (operation === "multiply") {
    return `${number1 * number2}`;
    } else if (operation === "divide") {
    return `${number1 / number2}`;
    } else {
    throw new Error("Invalid operation.");
    }
},
{
    name: "calculator",
    description: "Can perform mathematical operations.",
    schema: calculatorSchema,
}
);

const instanceWithTools = instance.bindTools([calculatorTool]);

const res = await instanceWithTools.invoke("What is 3 * 12");
console.log(res)

AIMessage {
  "id": "chat-d2214d0bdb794483a213b3211cf0d819",
  "content": "",
  "additional_kwargs": {
    "tool_calls": [
      {
        "id": "chatcmpl-tool-257f3d39532141b89178c2120f81f0cb",
        "type": "function",
        "function": "[Object]"
      }
    ]
  },
  "response_metadata": {
    "tokenUsage": {
      "completion_tokens": 38,
      "prompt_tokens": 177,
      "total_tokens": 215
    },
    "finish_reason": "tool_calls"
  },
  "tool_calls": [
    {
      "name": "calculator",
      "args": {
        "number1": 3,
        "number2": 12,
        "operation": "multiply"
      },
      "type": "tool_call",
      "id": "chatcmpl-tool-257f3d39532141b89178c2120f81f0cb"
    }
  ],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 177,
    "output_tokens": 38,
    "total_tokens": 215
  }
}

API reference

For detailed documentation of all IBM watsonx.ai features and configurations head to the API reference: API docs

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Popular Providers

General integrations

RAG integrations

Overview

Integration details

Model features

Setup

Credentials

IAM authentication

Bearer token authentication

IBM watsonx.ai software authentication

IAM authentication

Bearer token authentication

IBM watsonx.ai software authentication

Installation

Instantiation

Using model Gateway

Invocation

Streaming the model output

Tool calling

API reference

​Overview

​Integration details

​Model features

​Setup

​Credentials

​IAM authentication

​Bearer token authentication

​IBM watsonx.ai software authentication

​IAM authentication

​Bearer token authentication

​IBM watsonx.ai software authentication

​Installation

​Instantiation

​Using model Gateway

​Invocation

​Streaming the model output

​Tool calling

​API reference

Overview

Integration details

Model features

Setup

Credentials

IAM authentication

Bearer token authentication

IBM watsonx.ai software authentication

IAM authentication

Bearer token authentication

IBM watsonx.ai software authentication

Installation

Instantiation

Using model Gateway

Invocation

Streaming the model output

Tool calling

API reference