llama.cpp python library is a simple Python bindings for @ggerganov llama.cpp. This package provides:
  • Low-level access to C API via ctypes interface.
  • High-level Python API for text completion
    • OpenAI-like API
    • LangChain compatibility
    • LlamaIndex compatibility
  • OpenAI compatible web server
    • Local Copilot replacement
    • Function Calling support
    • Vision API support
    • Multiple Models

​
Installation and Setup

  • Install the Python package 
    pip install llama-cpp-python
  • Download one of the supported models and convert them to the llama.cpp format per the instructions

​
Chat models

See a usage example. 
from langchain_community.chat_models import ChatLlamaCpp

​
LLMs

See a usage example. 
from langchain_community.llms import LlamaCpp

​
Embedding models

See a usage example. 
from langchain_community.embeddings import LlamaCppEmbeddings