Run self-hosted Sandboxes in a separate cluster

​

Decide when to use a separate cluster

• Your LangSmith cluster cannot host sandbox nodes: The sandbox runtime needs KVM-capable nodes and privileged pods. For example, GKE Autopilot clusters cannot run the runtime, so a LangSmith installation on Autopilot needs a separate GKE Standard cluster for Sandboxes.
• You want to isolate sandbox workloads: Sandboxes run user code on privileged hosts. A separate cluster keeps those nodes away from the cluster that runs LangSmith and holds its application secrets. The runtime receives only the two secret values it needs.

​

Understand the topology

​

Check the requirements

​

Every cloud

• Nodes: KVM-capable linux/amd64 nodes, tainted sandbox.langsmith.com/host=true:NoSchedule and labeled to match sandboxHost.deployment.nodeSelector. The node selector has no default, and the chart fails to render without it. The default toleration matches the taint above; for other taints, override sandboxHost.deployment.tolerations. For node-level requirements, see the Enable Sandboxes prerequisites and Node requirements for bare-metal and on-premises clusters.
• JuiceFS metadata Redis: Use maxmemory-policy noeviction. Require AUTH, TLS, or IAM authentication whenever another cluster can reach it. The storage backends and bucket formats in the Enable Sandboxes prerequisites apply to juicefs.storage and juicefs.bucket.
• LangSmith hostname: Set config.hostname in the langsmith release to an address the sandbox cluster can reach. With an external runtime, the chart fails to render without it, or with a loopback or in-cluster Service address.
• config.endpoint: Set the runtime chart’s config.endpoint to https://<LangSmith hostname>[/<base path>]/api, the externally reachable LangSmith API including the /api prefix.
• Private CA: If a private CA signs the LangSmith API certificate, the host ingress certificate, or an HTTPS egress proxy certificate, add that CA to the runtime chart’s config.customCa.
• Network: Open the paths in Configure the network.
• Secrets: Create the Secret in Create the runtime Secret.

​

Amazon EKS

• Identity: An IAM OIDC provider for the sandbox cluster, and an IAM role that trusts system:serviceaccount:<namespace>:<service-account-name>. Set sandboxHost.serviceAccount.name so the subject stays stable. Set workloadIdentity.provider: aws and workloadIdentity.aws.roleArn.
• Storage grants: s3:GetObject, s3:PutObject, s3:DeleteObject, and s3:AbortMultipartUpload on arn:aws:s3:::<bucket>/<juicefs.name>/*, and s3:ListBucket on arn:aws:s3:::<bucket> with the condition s3:prefix like <juicefs.name>/*. A bucket that only JuiceFS uses needs no condition.
• Nodes: Instance types that expose KVM, such as bare-metal instances.
• Cache: The JuiceFS cache defaults to /var/cache/juicefs on the root volume. On instance types with local NVMe, such as m5d.metal, mount the drives and point juicefs.hostMount.cacheDirs at them.

​

Google Kubernetes Engine

• Cluster: A Standard cluster, not Autopilot, with a node pool on UBUNTU_CONTAINERD and nested virtualization (N2, N1, C3, or C4). Set the node pool’s OAuth scope to cloud-platform, so IAM grants decide access.
• Identity: Set workloadIdentity.provider: gcp. sandbox-host uses the host network, and the chart runs the JuiceFS formatter there too, so both authenticate as the node pool’s service account and need no Workload Identity binding. Leave juicefsFormatJob.nodeSelector empty, so the formatter runs on the sandbox node pool.
• Storage grants: Use a bucket that only JuiceFS uses, with uniform bucket-level access, and grant the node pool’s service account roles/storage.objectAdmin on it. If JuiceFS must share a bucket, turn on uniform bucket-level access and use the condition resource.name.startsWith("projects/_/buckets/<bucket>/objects/<juicefs.name>/") || resource.type == "storage.googleapis.com/Bucket". A prefix-only condition blocks the list calls JuiceFS makes. The second clause lets sandbox nodes list the name and metadata of every object in the bucket.
• Redis: With juicefs.redis.iamAuthProvider: gcp, use Memorystore for Redis Cluster and grant the node pool’s service account roles/redis.dbConnectionUser.
• Cache: Point juicefs.hostMount.cacheDirs at node-local SSD.
• Namespace: If the sandbox cluster shares a Google Cloud project with the LangSmith cluster and has Workload Identity enabled, install the runtime in a namespace other than the LangSmith one. Workload Identity trusts a namespace and ServiceAccount name across every cluster in a project.

​

Azure Kubernetes Service

• Nodes: A VM size that supports nested virtualization.
• Identity: OIDC issuer and Workload Identity enabled, and a federated credential with the subject system:serviceaccount:<namespace>:<service-account-name>. Set sandboxHost.serviceAccount.name so the subject stays stable. Set workloadIdentity.provider: azure and workloadIdentity.azure.clientId, plus tenantId if it differs from the cluster’s tenant. The chart adds the azure.workload.identity/use label to sandbox-host and the formatter.
• Storage grants: Storage Blob Data Contributor on the container. Set juicefs.storage: wasb and juicefs.storageAccountName.
• Redis: Azure Managed Redis on database 0 over rediss://, with the NoEviction eviction policy set on the database. JuiceFS cannot change it.
• Cache: Point juicefs.hostMount.cacheDirs at the temporary disk under /mnt.

​

Configure the network

• Load balancer allowlists: If the LangSmith load balancer restricts source addresses, allow the address sandbox node traffic arrives from. For an internal load balancer, that is usually the sandbox nodes’ private addresses. For an internet-facing one, it is usually the public address of the NAT gateway that the sandbox nodes use.
• Egress filtering: If a firewall filters egress from the sandbox cluster, allow the LangSmith hostname.
• DNS: The sandbox nodes must resolve the LangSmith hostname. An address that resolves only in the LangSmith cluster’s private DNS does not work. For an internal load balancer, add the record to a private DNS zone the sandbox network uses, before you install the runtime: a lookup that fails at startup can stay cached for several minutes.

​

Use the host ingress

1. In the runtime values, enable sandboxHost.ingress with a hostname and TLS:
   sandboxHost:
     ingress:
       enabled: true
       ingressClassName: "<ingress-class>"
       hostname: "sandbox-hosts.example.com"
       tls:
         - secretName: "sandbox-hosts-tls"
           hosts:
             - "sandbox-hosts.example.com"
   
2. In the langsmith values, set the ingress origin, with no path or trailing slash:
   sandboxes:
     external:
       hostIngressEndpoint: "https://sandbox-hosts.example.com"
   

• Clients: The LangSmith platform-backend and ingest-queue pods connect to the ingress, and so do the sandbox hosts, which dial it for sandbox links. If the ingress restricts source addresses, allow both. The sandbox hosts must also resolve the ingress hostname and trust its certificate. If a private CA signs it, add the CA to the runtime chart’s customCa and to the langsmith chart’s config.customCa.
• Health checks: Load balancers check the sandbox-host pods directly, and many check / or the Ingress path by default, which returns 404. Set the health check path to /readyz. With the AWS Load Balancer Controller, add alb.ingress.kubernetes.io/healthcheck-path: /readyz and alb.ingress.kubernetes.io/target-type: ip to sandboxHost.ingress.annotations, because the Service is ClusterIP. On GKE, leave ingressClassName empty, set the kubernetes.io/ingress.class: gce annotation, and attach a BackendConfig with healthCheck.requestPath: /readyz and healthCheck.port: 19190 through sandboxHost.ingress.service.annotations (cloud.google.com/backend-config).
• Timeouts: LangSmith waits up to 310 seconds for an exec request. Exec streams, tunnels, sandbox links, and browser sessions are long-lived WebSocket or server-sent event connections. Set the controller’s read, idle, and backend timeouts above 310 seconds, and higher if long sessions must stay open. Defaults of 30 to 60 seconds cut them off. On GKE, set timeoutSec in the same BackendConfig.
• Request size: File uploads go through the ingress. Raise the controller’s body size limit to at least your largest upload. On ingress-nginx, set nginx.ingress.kubernetes.io/proxy-body-size, which defaults to 1m.
• WebSockets: The controller must allow WebSocket upgrades on Ingress routes. Azure Application Gateway for Containers supports WebSockets only through the Gateway API, so it cannot serve this Ingress. Use another controller, or leave sandboxHost.ingress disabled and route the two paths to port 19190 of the sandbox-host pods with your own Service and Gateway API HTTPRoute.

​

Create the runtime Secret

• Sandbox service-auth secret: Signs calls between the control plane and the sandbox hosts. Store it under sandbox_service_auth_secret, or the key that serviceAuth.secretKey names.
• Callback signing JWK: The Ed25519 private JWK that signs sandbox callbacks. Store it under sandbox_callback_signing_jwk, or the key that config.callbackSigningJwkSecretKey names.

apiVersion: v1
kind: Secret
metadata:
  name: langsmith-sandbox-runtime
  namespace: langsmith-sandbox
type: Opaque
stringData:
  sandbox_service_auth_secret: '<same-value-as-the-control-plane>'
  sandbox_callback_signing_jwk: '<same-ed25519-private-jwk-as-the-control-plane>'

• Chart-managed Secret: Set sandboxes.serviceAuth.secret. It must differ from config.apiKeySalt.
• config.existingSecretName: Add a key with the new value to your Secret, and name it in sandboxes.serviceAuth.secretKey.

​

Generate a callback signing JWK

openssl genpkey -algorithm ED25519 -out callback-signing.pem
d=$(openssl pkey -in callback-signing.pem -outform DER | tail -c 32 | base64 | tr '+/' '-_' | tr -d '=\n')
x=$(openssl pkey -in callback-signing.pem -pubout -outform DER | tail -c 32 | base64 | tr '+/' '-_' | tr -d '=\n')
printf '{"kty":"OKP","crv":"Ed25519","alg":"EdDSA","use":"sig","kid":"sandbox-callback-%s","x":"%s","d":"%s"}\n' "$(openssl rand -hex 8)" "$x" "$d"
rm callback-signing.pem

​

Install with Helm

config:
  hostname: "langsmith.example.com"

sandboxes:
  enabled: true
  external:
    enabled: true
    # Only when you use the host ingress:
    # hostIngressEndpoint: "https://sandbox-hosts.example.com"
  # Required: a dedicated service-auth value, never api_key_salt. With
  # config.existingSecretName, set serviceAuth.secretKey to its key instead.
  serviceAuth:
    secret: "<random-secret>"

helm upgrade -i langsmith langchain/langsmith \
  --kube-context <langsmith-context> \
  --namespace <langsmith-namespace> \
  --version <langsmith-chart-version> \
  --values langsmith_config.yaml \
  --wait

kubectl --context <sandbox-context> create namespace langsmith-sandbox
kubectl --context <sandbox-context> apply -f langsmith-sandbox-runtime-secret.yaml

config:
  endpoint: "https://langsmith.example.com/api"
  existingSecretName: "langsmith-sandbox-runtime"

workloadIdentity:
  provider: aws
  aws:
    roleArn: "<role_arn>"

juicefs:
  name: "sandbox-juicefs"
  storage: "s3"
  bucket: "https://bucket-name.s3.us-west-2.amazonaws.com"
  redis:
    metaURL: "rediss://<redis-host>:6379/0"

sandboxHost:
  enabled: true
  serviceAccount:
    name: "langsmith-sandbox-host"
  deployment:
    nodeSelector:
      kubernetes.io/arch: "amd64"
      sandbox.langsmith.com/host: "true"

config:
  endpoint: "https://langsmith.example.com/api"
  existingSecretName: "langsmith-sandbox-runtime"

workloadIdentity:
  provider: gcp

juicefs:
  name: "sandbox-juicefs"
  storage: "gs"
  bucket: "gs://bucket-name"
  redis:
    # Memorystore with AUTH. With in-transit encryption, use rediss:// on port 6378.
    metaURL: "redis://:<auth-string>@<redis-host>:6379/0"
  # Without local SSD on the node pool, omit hostMount to keep the cache on the
  # boot disk (/var/cache/juicefs).
  hostMount:
    cacheDirs:
      - "<node-local-ssd-path>/juicefs"

sandboxHost:
  enabled: true
  deployment:
    nodeSelector:
      kubernetes.io/arch: "amd64"
      sandbox.langsmith.com/host: "true"

config:
  endpoint: "https://langsmith.example.com/api"
  existingSecretName: "langsmith-sandbox-runtime"

workloadIdentity:
  provider: azure
  azure:
    clientId: "<client_id>"

juicefs:
  name: "sandbox-juicefs"
  storage: "wasb"
  bucket: "https://<container>.blob.core.windows.net"
  storageAccountName: "<storage-account-name>"
  redis:
    # Azure Managed Redis with an access key. With Microsoft Entra authentication,
    # drop the key and set iamAuthProvider: azure.
    metaURL: "rediss://:<url-encoded-access-key>@<redis-host>:10000/0"
  hostMount:
    cacheDirs:
      - "/mnt/juicefs-cache"

sandboxHost:
  enabled: true
  serviceAccount:
    # Must match the subject of the managed identity's federated credential.
    name: "langsmith-sandbox-host"
  deployment:
    nodeSelector:
      kubernetes.io/arch: "amd64"
      sandbox.langsmith.com/host: "true"

helm repo update
helm search repo langchain/langsmith-sandbox --versions --devel

helm upgrade -i langsmith-sandbox langchain/langsmith-sandbox \
  --kube-context <sandbox-context> \
  --namespace langsmith-sandbox \
  --version <matching-sandbox-chart-version> \
  --values langsmith_sandbox_config.yaml \
  --wait

​

Install with Argo CD or Flux

• Order: Deploy the langsmith release first. Add the runtime after the langsmith application has synced and is Healthy (Flux: the HelmRelease is Ready).
• Versions: Pin langsmith-sandbox to the exact chart version whose appVersion matches. Never use a range, *, or *-0, and never omit version in a Flux HelmRelease.
• Proxy CA with Argo CD: Set proxyCa.mode: existingSecret and point proxyCa.existingSecretName at a TLS Secret you create in the runtime namespace, with keys tls.crt and tls.key, for example kubectl create secret tls <name> --cert=ca.crt --key=ca.key with a CA certificate you generate. The default generatedSecret mode reuses its CA through Helm lookup, which Argo CD and other tools that render without cluster access cannot run, so each render produces a new CA. Flux runs Helm in the cluster, so the default mode works there.
• Install notes: Argo CD and Flux do not show Helm install notes. The rules that the notes print are on this page.
• Back to the Helm CLI: To stop managing the runtime with Flux, set spec.suspend: true on its HelmRelease before you delete it; deleting an active HelmRelease uninstalls the release. Helm 4 applies changes server-side, so the first helm upgrade afterwards fails with conflicts on fields that Flux’s helm-controller owns. Run that one upgrade with --force-conflicts.

​

Verify the installation

helm list --kube-context <langsmith-context> -n <langsmith-namespace> --filter '^langsmith$'
helm list --kube-context <sandbox-context> -n langsmith-sandbox --filter '^langsmith-sandbox$'

kubectl --context <langsmith-context> -n <langsmith-namespace> get deployment langsmith-platform-backend \
  -o jsonpath='{.spec.template.spec.containers[?(@.name=="platform-backend")].image}{"\n"}'
kubectl --context <sandbox-context> -n langsmith-sandbox get deployment langsmith-sandbox-sandbox-host \
  -o jsonpath='{.spec.template.spec.containers[?(@.name=="sandbox-host")].image}{"\n"}'

kubectl --context <sandbox-context> -n langsmith-sandbox rollout status deployment/langsmith-sandbox-sandbox-host
kubectl --context <sandbox-context> -n langsmith-sandbox get pods

kubectl --context <langsmith-context> -n <langsmith-namespace> get secret <langsmith-app-secret> \
  -o jsonpath='{.data.<service-auth-key>}' | base64 -d | openssl dgst -sha256
kubectl --context <sandbox-context> -n langsmith-sandbox get secret langsmith-sandbox-runtime \
  -o jsonpath='{.data.sandbox_service_auth_secret}' | base64 -d | openssl dgst -sha256

​

Run one runtime per installation

• Its elected sandbox-host pod turns not ready about 30 seconds after its first refused report and stays not ready. A helm install --wait or kubectl rollout status that finishes within the first seconds can still report success, so check the pod again after a minute.
• Its sandbox-host logs repeat the platform refused this sandbox runtime because another runtime already reports to the same LangSmith install, with the error code sandbox_runtime_conflict, every 15 seconds.
• LangSmith logs refused host report from a second sandbox runtime, with the instance IDs of the accepted and the refused runtime.

​

Watch for a release mismatch

​

Move an existing installation to a separate cluster

kubectl --context <langsmith-context> -n <langsmith-namespace> get secret smithbox-proxy-ca -o json \
  | jq '{apiVersion, kind, type, metadata: {name: .metadata.name, namespace: "langsmith-sandbox"}, data: {"tls.crt": .data["tls.crt"], "tls.key": .data["tls.key"]}}' \
  | kubectl --context <sandbox-context> apply -f -

kubectl --context <langsmith-context> -n <langsmith-namespace> wait --for=delete pod \
  -l app.kubernetes.io/component=langsmith-sandbox-host --timeout=15m

​

Rotate the service-auth secret

1. On both sides, add the new value as the previous secret. In the langsmith chart, set sandboxes.serviceAuth.previousSecret, or with config.existingSecretName, add a key and name it in sandboxes.serviceAuth.previousSecretKey. In the runtime chart, add a key to the runtime Secret and name it in serviceAuth.previousSecretKey.
2. On both sides, make the new value current and the old value the previous secret. If you edit an existing Secret in place, this step changes no chart values, so restart the Deployments that read the Secret instead of applying the releases.
3. On both sides, remove the previous secret. With an existing Secret, first unset the previous-secret key setting and apply the release, then delete the key from the Secret: pods that reference a deleted key cannot start.

​

Route egress through a forward proxy

• Proxy URL: Add the URL to the runtime Secret under the key sandbox_egress_proxy_url, or the key that authProxy.httpProxy.urlSecretKey names. The URL can carry credentials, so the chart reads it only from the Secret. Without the key, Sandboxes reach destinations directly.
• Bypass list: Set authProxy.httpProxy.noProxy.
• Private CA: For an HTTPS proxy with a private CA, add the CA to config.customCa.

​

Uninstall

helm uninstall langsmith-sandbox --kube-context <sandbox-context> -n langsmith-sandbox --wait

​

See also

• Enable Sandboxes
• Sandbox architecture and lifecycle
• Operate self-hosted Sandboxes
• Upgrade an installation
• Release policy