> ## Documentation Index > Fetch the complete documentation index at: https://docs.langchain.com/llms.txt > Use this file to discover all available pages before exploring further. # IBM watsonx.ai integration > Integrate with the IBM watsonx.ai LLM using LangChain Python. > [`WatsonxLLM`](https://ibm.github.io/watsonx-ai-python-sdk/fm_extensions.html#langchain) is a wrapper for IBM [watsonx.ai](https://www.ibm.com/products/watsonx-ai) foundation models. This example shows how to communicate with `watsonx.ai` models using `LangChain`. ## Overview ### Integration details | Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/llms/ibm/) | Downloads | Version | | :-------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------ | :---: | :----------: | :----------------------------------------------------------------: | :--------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------: | | [`WatsonxLLM`](https://reference.langchain.com/python/integrations/langchain_ibm/WatsonxLLM/) | [`langchain-ibm`](https://reference.langchain.com/python/integrations/langchain_ibm/) | ❌ | ❌ | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ibm?style=flat-square\&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ibm?style=flat-square\&label=%20) | ## Setup To access IBM watsonx.ai models you'll need to create an IBM watsonx.ai account, get an API key, and install the `langchain-ibm` integration package. ### Credentials The cell below defines the credentials required to work with watsonx Foundation Model inferencing. **Action:** Provide the IBM Cloud user API key. For details, see [Managing user API keys](https://cloud.ibm.com/docs/account?topic=account-userapikey\&interface=ui). ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} import os from getpass import getpass watsonx_api_key = getpass() os.environ["WATSONX_APIKEY"] = watsonx_api_key ``` Additionally you are able to pass additional secrets as an environment variable. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} import os os.environ["WATSONX_URL"] = "your service instance url" os.environ["WATSONX_TOKEN"] = "your token for accessing the CLOUD or CPD cluster" os.environ["WATSONX_PASSWORD"] = "your password for accessing the CPD cluster" os.environ["WATSONX_USERNAME"] = "your username for accessing the CPD cluster" os.environ["WATSONX_INSTANCE_ID"] = "your instance_id for accessing the CPD cluster" ``` ### Installation The LangChain IBM integration lives in the `langchain-ibm` package: ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} !pip install -qU langchain-ibm ``` ## Instantiation You might need to adjust model `parameters` for different models or tasks. For details, refer to [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#metanames.GenTextParamsMetaNames). ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} parameters = { "decoding_method": "sample", "max_new_tokens": 100, "min_new_tokens": 1, "temperature": 0.5, "top_k": 50, "top_p": 1, } ``` Initialize the `WatsonxLLM` class with previously set parameters. **Note**: * To provide context for the API call, you must add `project_id` or `space_id`. For more information see [documentation](https://www.ibm.com/docs/en/watsonx-as-a-service?topic=projects). * Depending on the region of your provisioned service instance, use one of the URLs described in the [Watsonx Python SDK cloud setup guide](https://ibm.github.io/watsonx-ai-python-sdk/setup_cloud.html#authentication). In this example, we’ll use the `project_id` and Dallas URL. You need to specify `model_id` that will be used for inferencing. All available models you can find in [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#TextModels). ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} from langchain_ibm import WatsonxLLM watsonx_llm = WatsonxLLM( model_id="ibm/granite-13b-instruct-v2", url="https://us-south.ml.cloud.ibm.com", project_id="PASTE YOUR PROJECT_ID HERE", params=parameters, ) ``` Alternatively you can use Cloud Pak for Data credentials. For details, see [documentation](https://ibm.github.io/watsonx-ai-python-sdk/setup_cpd.html). ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} watsonx_llm = WatsonxLLM( model_id="ibm/granite-13b-instruct-v2", url="PASTE YOUR URL HERE", username="PASTE YOUR USERNAME HERE", password="PASTE YOUR PASSWORD HERE", instance_id="openshift", version="4.8", project_id="PASTE YOUR PROJECT_ID HERE", params=parameters, ) ``` Instead of `model_id`, you can also pass the `deployment_id` of the previously tuned model. The entire model tuning workflow is described in [Working with TuneExperiment and PromptTuner](https://ibm.github.io/watsonx-ai-python-sdk/pt_tune_experiment_run.html). ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} watsonx_llm = WatsonxLLM( deployment_id="PASTE YOUR DEPLOYMENT_ID HERE", url="https://us-south.ml.cloud.ibm.com", project_id="PASTE YOUR PROJECT_ID HERE", params=parameters, ) ``` For certain requirements, there is an option to pass the IBM's [`APIClient`](https://ibm.github.io/watsonx-ai-python-sdk/base.html#apiclient) object into the `WatsonxLLM` class. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} from ibm_watsonx_ai import APIClient api_client = APIClient(...) watsonx_llm = WatsonxLLM( model_id="ibm/granite-13b-instruct-v2", watsonx_client=api_client, ) ``` You can also pass the IBM's [`ModelInference`](https://ibm.github.io/watsonx-ai-python-sdk/fm_model_inference.html) object into the `WatsonxLLM` class. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} from ibm_watsonx_ai.foundation_models import ModelInference model = ModelInference(...) watsonx_llm = WatsonxLLM(watsonx_model=model) ``` ## Invocation To obtain completions, you can call the model directly using a string prompt. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} # Calling a single prompt watsonx_llm.invoke("Who is man's best friend?") ``` ```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} "Man's best friend is his dog. Dogs are man's best friend because they are always there for you, they never judge you, and they love you unconditionally. Dogs are also great companions and can help reduce stress levels. " ``` ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} # Calling multiple prompts watsonx_llm.generate( [ "The fastest dog in the world?", "Describe your chosen dog breed", ] ) ``` ```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound. Greyhounds can run up to 45 mph, which is about the same speed as a Usain Bolt.', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Labrador Retriever is a breed of retriever that was bred for hunting. They are a very smart breed and are very easy to train. They are also very loyal and will make great companions. ', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 82, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': None}, run=[RunInfo(run_id=UUID('750b8a0f-8846-456d-93d0-e039e95b1276')), RunInfo(run_id=UUID('aa4c2a1c-5b08-4fcf-87aa-50228de46db5'))], type='LLMResult') ``` ## Streaming the model output You can stream the model output. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} stream = watsonx_llm.stream_events( "Describe your favorite breed of dog and why it is your favorite.", version="v3", ) for token in stream.text: print(token, end="") ``` ```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} My favorite breed of dog is a Labrador Retriever. They are my favorite breed because they are my favorite color, yellow. They are also very smart and easy to train. ``` ## Chaining Create `PromptTemplate` objects which will be responsible for creating a random question. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} from langchain_core.prompts import PromptTemplate template = "Generate a random question about {topic}: Question: " prompt = PromptTemplate.from_template(template) ``` Provide a topic and run the chain. ```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} llm_chain = prompt | watsonx_llm topic = "dog" llm_chain.invoke(topic) ``` ```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} 'What is the origin of the name "Pomeranian"?' ``` *** ## API reference For detailed documentation of all `WatsonxLLM` features and configurations head to the [API reference](https://reference.langchain.com/python/integrations/langchain_ibm/WatsonxLLM/). ***
[Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers. [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/llms/ibm_watsonx.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).