Anyscale is a platform to run, fine tune and scale LLMs via production-ready APIs. Anyscale Endpoints serve many open-source models in a cost-effective way.
Anyscale also provides an example how to setup LangChain with Anyscale for advanced chat agents.

​
Installation and Setup

  • Get an Anyscale Service URL, route and API key and set them as environment variables (ANYSCALE_SERVICE_URL,ANYSCALE_SERVICE_ROUTE, ANYSCALE_SERVICE_TOKEN).
  • Please see the Anyscale docs for more details.
We have to install the openai package: 
pip install openai

​
LLM

See a usage example. 
from langchain_community.llms.anyscale import Anyscale

​
Chat Models

See a usage example. 
from langchain_community.chat_models.anyscale import ChatAnyscale

​
Embeddings

See a usage example. 
from langchain_community.embeddings import AnyscaleEmbeddings