Together AI offers an API to query 50+ leading open-source models in a couple lines of code. This guide will help you getting started with ChatTogetherAI chat models. For detailed documentation of all ChatTogetherAI features and configurations head to the API reference.

​
Overview

​
Integration details

ClassPackageLocalSerializablePY supportDownloadsVersion
ChatTogetherAI@langchain/community❌✅✅NPM - DownloadsNPM - Version

​
Model features

See the links in the table headers below for guides on how to use specific features.
Tool callingStructured outputJSON modeImage inputAudio inputVideo inputToken-level streamingToken usageLogprobs
✅✅✅✅✅✅✅✅✅

​
Setup

To access ChatTogetherAI models you’ll need to create a Together account, get an API key here, and install the @langchain/community integration package.

​
Credentials

Head to api.together.ai to sign up to TogetherAI and generate an API key. Once you’ve done this set the TOGETHER_AI_API_KEY environment variable: 
export TOGETHER_AI_API_KEY="your-api-key"
If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below: 
# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

​
Installation

The LangChain ChatTogetherAI integration lives in the @langchain/community package: 
import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";
<IntegrationInstallTooltip></IntegrationInstallTooltip>

<Npm2Yarn>
  @langchain/community @langchain/core
</Npm2Yarn>

​
Instantiation

Now we can instantiate our model object and generate chat completions: 
import { ChatTogetherAI } from "@langchain/community/chat_models/togetherai"

const llm = new ChatTogetherAI({
    model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
    temperature: 0,
    // other params...
})

​
Invocation

const aiMsg = await llm.invoke([
    [
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ],
    ["human", "I love programming."],
])
aiMsg

AIMessage {
  "id": "chatcmpl-9rT9qEDPZ6iLCk6jt3XTzVDDH6pcI",
  "content": "J'adore la programmation.",
  "additional_kwargs": {},
  "response_metadata": {
    "tokenUsage": {
      "completionTokens": 8,
      "promptTokens": 31,
      "totalTokens": 39
    },
    "finish_reason": "stop"
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 31,
    "output_tokens": 8,
    "total_tokens": 39
  }
}

console.log(aiMsg.content)

J'adore la programmation.

​
Chaining

We can chain our model with a prompt template like so: 
import { ChatPromptTemplate } from "@langchain/core/prompts"

const prompt = ChatPromptTemplate.fromMessages(
    [
        [
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ],
        ["human", "{input}"],
    ]
)

const chain = prompt.pipe(llm);
await chain.invoke(
    {
        input_language: "English",
        output_language: "German",
        input: "I love programming.",
    }
)

AIMessage {
  "id": "chatcmpl-9rT9wolZWfJ3xovORxnkdf1rcPbbY",
  "content": "Ich liebe das Programmieren.",
  "additional_kwargs": {},
  "response_metadata": {
    "tokenUsage": {
      "completionTokens": 6,
      "promptTokens": 26,
      "totalTokens": 32
    },
    "finish_reason": "stop"
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 26,
    "output_tokens": 6,
    "total_tokens": 32
  }
}
Behind the scenes, TogetherAI uses the OpenAI SDK and OpenAI compatible API, with some caveats:

​
API reference

For detailed documentation of all ChatTogetherAI features and configurations head to the API reference.