Ctranslate2 integrations

Documentation Index

Fetch the complete documentation index at: https://docs.langchain.com/llms.txt

Use this file to discover all available pages before exploring further.

CTranslate2 is a C++ and Python library for efficient inference with Transformer models. The project implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering, etc., to accelerate and reduce the memory usage of Transformer models on CPU and GPU. A full list of features and supported models is included in the project’s repository. To start, please check out the official quickstart guide.

​

Installation and setup

pip install ctranslate2

​

LLMs

from langchain_community.llms import CTranslate2