Skip to main content
LangChain operates your data plane after provisioning, including upgrades, scaling, monitoring, and security patching. Most maintenance is non-disruptive and requires no action from you. Operations fall into three groups:

LangChain-managed operations

LangChain performs the following operations on every data plane, with no action required from you. To change some defaults, such as high availability or instance sizes, contact the LangChain team.

Autoscaling

LangChain scales your data plane as load changes, so you do not need to size or tune it:
  • Stateless LangSmith services: Scaled horizontally by a horizontal pod autoscaler, with Karpenter provisioning and removing node capacity as pod demand changes.
  • Databases: LangChain sizes and scales RDS and ElastiCache.

High availability

Data planes are provisioned for high availability by default. The deployment spans multiple availability zones in your region, and RDS and ElastiCache run Multi-AZ with automatic failover.
To run a data plane without high availability, contact the LangChain team.

Instance sizes

Data planes are provisioned with default instance sizes for the databases and node groups. To run a data plane on smaller instances, contact the LangChain team.

LangSmith upgrades

LangChain upgrades the LangSmith version in your data plane daily. Upgrades are rolling, replica-by-replica, so there is no whole-service downtime. Two mechanisms keep upgrades free of downtime:
  • Horizontal pod autoscaling: LangSmith services run multiple replicas, sized by a horizontal pod autoscaler, so pods are replaced a few at a time while the remaining replicas continue serving traffic.
  • Pod disruption budgets: A pod disruption budget caps how many replicas of a service can be unavailable at once, so neither a rolling update nor a node drain takes a service below the capacity it needs to stay available.

Supporting service upgrades

Background services such as Istio and KEDA are upgraded on a regular cadence. Data planes that still run ClickHouse also receive ClickHouse operator upgrades. These upgrades are non-intrusive, and no service interruption is expected.

Kubernetes cluster upgrades

The EKS cluster that hosts your data plane requires periodic upgrades to maintain security, compatibility, and access to new features. LangChain performs these upgrades proactively, ahead of AWS end-of-support dates, in the data plane’s maintenance window. Cluster upgrades fall into two types:
  • Control plane upgrades: Cover the Kubernetes API server and the components AWS manages alongside it. These are transparent to your workloads and do not restart pods.
  • Node group upgrades: Roll nodes one by one with make-before-break semantics. New nodes are provisioned on the updated version, pods move onto them, and the old nodes are terminated once migration completes. Pod disruption budgets apply to the drain, so each service stays available, though you might see brief connection resets as individual pods restart. No data is lost.

Customer-managed operations

You configure the following operations yourself, through the LangSmith UI and API or in your own AWS account.

Data plane settings

To edit a data plane’s settings:
  1. Navigate to Settings > Data Planes.
  2. Find your data plane and select Settings.
  3. Open the tab for the setting you want to change, then save.
You can also use the Update data plane settings endpoint.

Maintenance window

The maintenance window is when LangChain runs operations on your data plane, such as EKS version upgrades and RDS database upgrades. Any maintenance that could cause downtime, such as a restart of RDS or ElastiCache, happens in this window.
  • The window lasts 2 hours, once a week. All times are in UTC.
  • Choose the day and the start time on the Maintenance tab.
  • The default is Sunday, 07:00–09:00 UTC (sun:07:00-sun:09:00).

TTL retention

A data plane’s TTL settings set the default retention periods for its traces. These values are defaults, not limits. A workspace that chooses one of the extended durations keeps that duration, even when it is longer than the data plane’s long-lived TTL. The blob bucket has an S3 lifecycle rule for each retention period, so trace payloads are deleted on the same schedule as their traces. Configure TTL retention on the TTL retention tab:
  • Enable TTL retention: On by default.
  • Short-lived TTL (days): The default retention period for base traces. The default is 14 days.
  • Long-lived TTL (days): The default retention period for extended traces. The default is 400 days. It must be greater than or equal to the short-lived TTL.
Each value must be a whole number greater than 0. Through the API, send ttl.short_days and ttl.long_days.

Ingress allowlist

On the Ingress tab, you can restrict which source IP ranges the load balancer accepts. Add up to 50 IPv4 CIDR ranges. The load balancer then accepts traffic on port 443 only from those ranges.
  • Empty list: The load balancer allows all sources.
  • Private load balancer: The data plane VPC CIDR is always allowed.
  • An API update replaces the whole list. Include every range you want to keep, or the load balancer stops accepting traffic from the ranges you leave out.
  • Do not edit the Network Load Balancer security group directly in AWS. LangSmith reconciles the data plane on a regular cadence and reverts any manual changes to managed resources.

Observability

You can deploy observability tooling into the cluster, such as a Datadog agent or an OpenTelemetry collector, and your own security monitoring agents, as long as they do not interfere with LangSmith workloads. LangSmith services emit logs, metrics, and traces the same way they do on self-hosted, so the same configuration applies. See Export LangSmith telemetry to your observability backend.

Cross-account access

A private BYOC data plane exposes its load balancer inside your data plane VPC. Applications in a different AWS account cannot reach it without an explicit cross-account network path. Any private network path between the consumer network and the data plane VPC works. LangChain does not require a specific method. Common options include:
  • AWS PrivateLink
  • VPC peering
  • Transit gateway
  • VPN
LangChain provisions two PrivateLink connections that carry traffic between the LangChain control plane and your data plane. There is no LangChain-managed endpoint service or network path for application traffic from your own networks. You create and manage that path yourself.
The steps below apply whether LangChain creates the VPC or you supply your own. To manage the VPC itself (its subnets, routing, gateways, and endpoints), use Bring your own VPC on AWS instead.

Allow consumer source ranges

If you restrict the load balancer’s source IP ranges, add the CIDR range of each consumer network to that list. A private load balancer always allows the data plane VPC CIDR, but not consumer ranges. Without them, the load balancer drops consumer traffic. To create the endpoint service in the data plane account:
  1. Identify the Network Load Balancer (NLB) for the data plane ingress. Find it in the AWS EC2 console under Load Balancers in your data plane account and region. Its name ends in -sm-istio.
  2. Create a VPC endpoint service backed by that NLB. Set acceptance mode to auto or manual depending on your security policy.
  3. Add each consumer AWS account as a principal allowed to create an interface endpoint for the service.
  4. Note the endpoint service name (for example, com.amazonaws.vpce.us-east-1.vpce-svc-xxxxxxxxxxxxxxxxx).
Do not modify the NLB itself. LangSmith reconciles the data plane on a regular cadence and reverts manual changes to managed resources. If the ingress mode or subnets change, LangChain coordinates any NLB updates with you.
To connect each consumer account:
  1. Create a VPC interface endpoint for the endpoint service name from the previous step, in the VPC where your application runs.
  2. Find the data plane hostname in the API URL shown under Settings > Data Planes.
  3. Create a private Route 53 hosted zone for that hostname, associated with the consumer VPC.
  4. Add an alias or CNAME record in that hosted zone pointing the hostname to the interface endpoint’s DNS name.
Your applications use the standard data plane API URL without any change. DNS resolves it to the interface endpoint in the consumer VPC.

Connect with VPC peering, a transit gateway, or a VPN

To peer a consumer VPC with the data plane VPC:
  1. In the data plane account, create a peering connection request from the data plane VPC to the consumer VPC.
  2. In the consumer account, accept the request.
  3. Add routes in both VPCs so traffic to each other’s CIDR range traverses the peering connection.
  4. Update any network ACLs or security groups that need to allow the peered ranges.
To use a transit gateway, attach both VPCs to a shared transit gateway and configure route tables so each VPC can reach the other. To use a VPN, attach the VPN to the data plane VPC, or to a transit gateway attached to it, and add routes so the consumer network can reach the data plane VPC CIDR range. These options route traffic, but they do not carry DNS. The data plane hostname resolves only inside the data plane VPC. To configure DNS in each consumer VPC:
  1. Create a private Route 53 hosted zone for the data plane hostname.
  2. Associate the hosted zone with the consumer VPC.
  3. Add an alias or CNAME record that points the hostname to the data plane load balancer’s DNS name.
For a consumer network outside AWS, forward DNS queries for the data plane hostname to a Route 53 Resolver inbound endpoint in the data plane VPC.

Auditing

  • LangSmith audit logs: Operations on data plane resources are recorded in the data plane and accessible at your data plane endpoint. Control plane actions, such as creating a user, are audited in the control plane. For how to access both, see Audit logs.
  • EKS audit logs: Sent to CloudWatch in your account, so you can monitor the commands that were run.
  • VPC flow logs: Recorded in an S3 bucket in your account. With BYOVPC, you configure VPC flow logs yourself.
Setting up alerts on those logs is your responsibility.

Break-glass access

The LangChain team has no standing access to your data, and can only manage the infrastructure of your data plane. When troubleshooting requires access to data, there are two paths:
  • You run the query: You run the query yourself. Then validate and sanitize the output before sharing it with LangChain.
  • Break-glass access: You grant a specific LangChain engineer temporary access to perform the operation. You can revoke that access at any time.

Grant access

Break-glass access uses the LangSmithBYOCBreakGlass IAM role that the langsmith-byoc-role Terraform module creates in your account. The role’s trust policy defaults to Effect: Deny. The role carries only eks:DescribeCluster permission at the IAM layer. Kubernetes-level access is controlled separately through EKS access entries that you create for the break-glass role in each target cluster and region. Three access levels are available: When a LangChain engineer requires access, they send you:
  • Their AWS IAM Identity Center user ID.
  • Their LangChain email address.
  • The requested access level.
  • The target cluster name and AWS region.
To grant access:
  1. Set the following inputs in your langsmith-byoc-role module and apply:
    The Identity Store user ID ensures that only the specified engineer’s credentials can assume the role. The source identity email improves readability in CloudTrail audit records.
  2. Share the module’s break_glass_role_arn output with the LangChain engineer.
  3. Set the values for the target cluster:
  4. Create the EKS access entry for the requested access level. Create only one access entry per principal per cluster.
Grant the lowest level that fits the problem. Activity appears in CloudTrail and in the EKS audit log in CloudWatch in your account.

Revoke access

Revoke access as soon as the engineer no longer needs it. Follow the steps in order: disabling trust first prevents the engineer from starting a new session after you revoke the active ones. To revoke access:
  1. Set allow_break_glass_access = false, remove the Identity Store user IDs and source identities, and apply:
    This blocks new sts:AssumeRole calls into the LangSmithBYOCBreakGlass role.
  2. Delete the EKS access entry in each cluster where you created it:
  3. In the IAM console, go to Roles > LangSmithBYOCBreakGlass > Revoke sessions, then select Revoke active sessions. Existing sessions stay valid until they expire, so revoking them closes access immediately.

See also