Setup guide (legacy)

Documentation Index

Fetch the complete documentation index at: https://docs.langchain.com/llms.txt

Use this file to discover all available pages before exploring further.

The hybrid model splits LangSmith infrastructure between LangChain’s cloud and yours:

• Control plane (LangSmith UI, APIs, and orchestration) runs in LangChain’s cloud, managed by LangChain.
• Data plane (your Agent Servers and agent workloads) runs in your cloud, managed by you.

This combines the convenience of a managed interface with the flexibility of running workloads in your own environment. When running LangSmith in a hybrid model, you authenticate with a LangSmith API key.

​

Workflow

1. Use the langgraph-cli or Studio to test your graph locally.
2. Build a Docker image using the langgraph build command.
3. Deploy your Agent Server from the control plane UI.

​

Architecture

​

Compute platforms

• Kubernetes: Hybrid supports running the data plane on any Kubernetes cluster.

​

Egress to LangSmith and the control plane

In the hybrid deployment model, your self-hosted data plane will send network requests to the control plane to poll for changes that need to be implemented in the data plane. Traces from data plane deployments also get sent to the LangSmith instance integrated with the control plane. This traffic to the control plane is encrypted, over HTTPS. The data plane authenticates with the control plane with a LangSmith API key. In order to enable this egress, you may need to update internal firewall rules or cloud resources (such as Security Groups) to allow certain IP addresses.

​

Kubernetes setup

The following steps describe how to connect your self-hosted data plane to the managed LangSmith control plane.

​

Prerequisites

1. KEDA is installed on your cluster.
     helm repo add kedacore https://kedacore.github.io/charts
     helm install keda kedacore/keda --namespace keda --create-namespace
   
   KEDA is used to automatically scale the deployment system based on queue size.
2. A valid Ingress controller is installed on your cluster. For more information about configuring ingress for your deployment, refer to Create an ingress for installations. We highly recommend using the modern Gateway API in a production setup.
3. If you plan to have the listener watch multiple namespaces, you MUST use the Gateway API or an Istio Gateway instead of the standard ingress resource. A standard ingress resource can only route traffic to services in the same namespace, whereas a Gateway or Istio Gateway can route traffic to services across multiple namespaces.
4. You have slack space in your cluster for multiple deployments. Cluster-Autoscaler is recommended to automatically provision new nodes.
5. You will need to enable egress to two control plane URLs. The listener polls these endpoints for deployments. Use the pair that matches your LangSmith region.

LangSmith Deployment control plane:

Region
GCP US
GCP EU
GCP APAC
AWS US

LangSmith API:

Region
GCP US
GCP EU
GCP APAC
AWS US

​

Setup

1. Provide your LangSmith organization ID to us. Your LangSmith organization will be configured to deploy the data plane in your cloud.
2. Create a listener from the LangSmith UI. The Listener data model is configured for the actual “listener” application.
   1. In the left-hand navigation, select Deployments > Listeners.
   2. In the top-right of the page, select + Create Listener.
   3. Enter a unique Compute ID for the listener. The Compute ID is a user-defined identifier that should be unique across all listeners in the current LangSmith workspace. The Compute ID is displayed to end users when they are creating a new deployment. Ensure that the Compute ID provides context to the end user about where their Agent Server deployments will be deployed to. For example, a Compute ID can be set to k8s-cluster-name-dev-01. In this example, the name of the Kubernetes cluster is k8s-cluster-name, dev denotes that the cluster is reserved for “development” workloads, and 01 is a numerical suffix to reduce naming collisions.
   4. Enter one or more Kubernetes namespaces. Later, the “listener” application will be configured to deploy to each of these namespaces.
   5. In the top-right on the page, select Submit.
   6. After the listener is created, copy the listener ID. You will use it later when installing the actual “listener” application in the Kubernetes cluster (step 5).
   Important Creating a listener from the LangSmith UI does not install the “listener” application in the Kubernetes cluster.
3. A Helm chart is provided to install the necessary components in your Kubernetes cluster.
   • langgraph-dataplane-listener: This is a service that listens to LangChain’s control plane for changes to your deployments and creates/updates downstream CRDs. This is the “listener” application.
   • LangGraphPlatform CRD: A CRD for LangSmith Deployment. This contains the spec for managing an instance of a LangSmith Deployment.
   • langgraph-dataplane-operator: This operator handles changes to your LangSmith CRDs.
   • langgraph-dataplane-redis: A Redis instance is used by the langgraph-dataplane-listener to manage various tasks (mainly creating and deleting deployments).
4. Configure your langgraph-dataplane-values.yaml file.
     config:
       langsmithApiKey: "" # API Key of your Workspace
       langsmithWorkspaceId: "" # Workspace ID
       hostBackendUrl: "https://api.host.langchain.com" # Use the matching regional LangSmith Deployment control plane URL from the table above
       smithBackendUrl: "https://api.smith.langchain.com" # Use the matching regional LangSmith API URL from the table above
       langgraphListenerId: "" # Listener ID from Step 2f
       watchNamespaces: "" # comma-separated list of Kubernetes namespaces that the listener and operator will deploy to
       enableLGPDeploymentHealthCheck: true # enable/disable health check step for deployments
   
     ingress:
       hostname: "" # specify a hostname that will be configured for all deployments
   
     operator:
       enabled: true
       createCRDs: true # set this to `false` if the CRD has been previously installed in the current Kubernetes cluster
   
   • config.langsmithApiKey: The langgraph-listener deployment authenticates with LangChain’s LangGraph control plane API with the langsmithApiKey.
   • config.langsmithWorkspaceId: The langgraph-listener deployment is coupled to Agent Server deployments in the LangSmith workspace. In other words, the langgraph-listener deployment can only manage Agent Server deployments in the specified LangSmith workspace ID.
   • config.langgraphListenerId: In addition to being coupled with a LangSmith workspace, the langgraph-listener deployment is also coupled to a listener. When a new Agent Server deployment is created, it is automatically coupled to a langgraphListenerId. Specifying langgraphListenerId ensures that the langgraph-listener deployment can only manage Agent Server deployments that are coupled to langgraphListenerId.
   • config.watchNamespaces: A comma-separated list of Kubernetes namespaces that the langgraph-listener deployment will deploy to. This list should match the list of namespaces specified in step 2d.
   • config.enableLGPDeploymentHealthCheck: To disable the Agent Server health check, set this to false.
   • ingress.hostname: As part of the deployment workflow, the langgraph-listener deployment attempts to call the Agent Server health check endpoint (GET /ok) to verify that the application has started up correctly. A typical setup involves creating a shared DNS record or domain for Agent Server deployments. This is not managed by LangSmith. Once created, set ingress.hostname to the domain, which will be used to complete the health check.
   • operator.createCRDs: Set this value to false if the Kubernetes cluster already has the LangGraphPlatform CRD installed. During installation, an error will occur if the CRD is already installed. This situation may occur if multiple listeners are deployed on the same Kubernetes cluster.
5. Deploy langgraph-dataplane Helm chart.
     helm repo add langchain https://langchain-ai.github.io/helm/
     helm repo update
     helm upgrade -i langgraph-dataplane langchain/langgraph-dataplane --values langgraph-dataplane-values.yaml --wait --debug
   
6. If successful, you will see three services start up in your namespace.
     NAME                                            READY   STATUS              RESTARTS   AGE
     langgraph-dataplane-listener-6dd4749445-zjmr4   0/1     ContainerCreating   0          26s
     langgraph-dataplane-operator-6b88879f9b-t76gk   1/1     Running             0          26s
     langgraph-dataplane-redis-0                     1/1     Running             0          25s
   
   Your hybrid infrastructure is now ready to create deployments.

​

Configuring additional data planes in the same cluster

To create a data plane in a different namespace in the same cluster, repeat the above steps and pass a -n option to helm upgrade to specify a different namespace. When installing multiple data planes in the same cluster, it is very important to follow the rules below:

1. The config.watchNamespaces list should never intersect with other installations config.watchNamespaces. For example, if installation A is watching namespaces foo,bar, installation B cannot watch either foo or bar. Multiple operators or listeners watching the same namespace will lead to unexpected behavior. This means that multiple LangSmith workspaces cannot deploy to the same namespace! Please review the cluster organization section to understand this better.
2. It is required to use the Gateway API or an Istio Gateway. Relying on the standard ingress resource can cause conflicts with Ingress objects created by other data planes in the same cluster. Because behavior in these cases depends on the specific ingress controller, this may result in unpredictable or undesired outcomes.

​

Listeners

In the hybrid option, one or more “listener” applications can run depending on how your LangSmith workspaces and Kubernetes clusters are organized.

​

Kubernetes cluster organization

• One or more listeners can run in a Kubernetes cluster.
• A listener can deploy into one or more namespaces in that cluster.
• Multiple listeners cannot deploy to the same namespace.
• Cluster owners are responsible for planning listener layout and Agent Server deployments.

​

LangSmith workspace organization

• A workspace can be associated with one or more listeners.
• A listener can only be associated with one workspace. LangSmith workspace to listener is a one-to-many relationship.
• A workspace can only deploy to Kubernetes clusters where all of its listeners are deployed.

​

Use cases

Here are some common listener configurations (not strict requirements):

​

Each LangSmith workspace → separate Kubernetes cluster

• Cluster alpha runs workspace A
• Cluster beta runs workspace B

​

One cluster, one namespace per workspace

• Cluster alpha, namespace 1 runs workspace A
• Cluster alpha, namespace 2 runs workspace B

​

Separate clusters, with shared “dev” cluster

• Cluster alpha runs workspace A
• Cluster beta runs workspace B
• Cluster dev runs workspaces A and B
• Both workspaces have two listeners; cluster dev has two listener deployments

---