CompatibilityOnly available in web environments.
You can run LLMs directly in your web browser using LangChain’s WebLLM integration.

​
Setup

You’ll need to install the WebLLM SDK module to communicate with your local model.
See this section for general instructions on installing integration packages.
npm
npm install -S @mlc-ai/web-llm @langchain/community @langchain/core

​
Usage

Note that the first time a model is called, WebLLM will download the full weights for that model. This can be multiple gigabytes, and may not be possible for all end-users of your application depending on their internet connection and computer specs. While the browser will cache future invocations of that model, we recommend using the smallest possible model you can. We also recommend using a separate web worker when invoking and loading your models to not block execution. 
// Must be run in a web environment, e.g. a web worker

import { ChatWebLLM } from "@langchain/community/chat_models/webllm";
import { HumanMessage } from "@langchain/core/messages";

// Initialize the ChatWebLLM model with the model record and chat options.
// Note that if the appConfig field is set, the list of model records
// must include the selected model record for the engine.

// You can import a list of models available by default here:
// https://github.com/mlc-ai/web-llm/blob/main/src/config.ts
//
// Or by importing it via:
// import { prebuiltAppConfig } from "@mlc-ai/web-llm";
const model = new ChatWebLLM({
  model: "Phi-3-mini-4k-instruct-q4f16_1-MLC",
  chatOptions: {
    temperature: 0.5,
  },
});

await model.initialize((progress: Record<string, unknown>) => {
  console.log(progress);
});

// Call the model with a message and await the response.
const response = await model.invoke([
  new HumanMessage({ content: "What is 1 + 1?" }),
]);

console.log(response);

/*
AIMessage {
  content: ' 2\n',
}
*/
Streaming is also supported.

​
Example

For a full end-to-end example, check out this project.