Connect LangSmith Engine to GitHub

LangSmith Engine reads your source code to diagnose issues and opens pull requests with proposed fixes. It connects to GitHub through a GitHub App. This page covers connecting repositories in LangSmith Cloud and configuring your own GitHub App for a self-hosted deployment.

LangSmith Cloud

In LangSmith Cloud, Engine connects through a LangChain-managed GitHub App. You do not create or configure an app yourself. To connect your repositories:

In the LangSmith console, open a tracing project and go to the Engine tab.
Under Connect your agent’s code repository, click Connect GitHub and authorize the LangChain-managed GitHub App.
Install the app on the repositories Engine should access. Installing the app on a GitHub organization may require approval from a GitHub organization owner. If you are not an owner, GitHub sends the owner an installation request to approve before the app becomes available.
Select the connected repository in the GitHub Repository field on the Engine tab.

For the access and retention model of the managed app, see Engine security.

Self-hosted

In a self-hosted deployment, you create and manage your own GitHub App and pass its credentials to the LangSmith Helm chart.

Create a GitHub App

Create the app

Go to GitHub Settings > Developer settings > GitHub Apps and click New GitHub App.

GitHub App name: Any unique name, for example acme-langsmith-engine.
Homepage URL: Your LangSmith deployment URL, for example https://langsmith.example.com.
Where can this GitHub App be installed?: For most self-hosted deployments, select Only on this account. Select Any account only if you intend to distribute the app.

Set the callback URL

Add the following Callback URL, replacing <langsmith-host> with your LangSmith hostname:

https://<langsmith-host>/api-host/v1/integrations/forge/github/callback

Set the webhook URL and secret

Generate a random webhook secret of at least 32 bytes with your secret manager or another cryptographically secure generator. Use the same value in GitHub and your LangSmith secret store.Under Webhook, select Active and set the Webhook URL, replacing <langsmith-host> with your LangSmith hostname:

https://<langsmith-host>/api-host/v1/integrations/forge/github/webhook

Paste the generated value into Webhook secret.

Set repository permissions

Under Permissions > Repository permissions, grant the following:

Contents: Read and write.
Pull requests: Read and write.
Metadata: Read-only (automatically selected).

Under Subscribe to events, select no events. Engine does not require any event subscriptions.

Create the app and gather its values

Click Create GitHub App. GitHub supplies the following values on the app settings page:

Value	Where to find it	Environment variable
App ID	Numeric, at the top of the page	`FORGE_GITHUB_APP_ID`
Public link	For example, `https://github.com/apps/acme-langsmith-engine`	`FORGE_GITHUB_APP_PUBLIC_LINK`
Client ID	Under About	`FORGE_GITHUB_CLIENT_ID`
Client secret	Under Client secrets, click Generate a new client secret (shown once)	`FORGE_GITHUB_CLIENT_SECRET`
Private key	Under Private keys, click Generate a private key (downloads a `.pem` file)	`FORGE_GITHUB_APP_PEM`

Generate a state JWT secret

LangSmith signs short-lived OAuth state tokens with an HMAC key. Generate a random secret of at least 32 bytes with your secret manager or another cryptographically secure generator. GitHub does not provide this value.This is FORGE_GITHUB_STATE_JWT_SECRET. Generate it separately, and do not reuse the webhook secret or any other credential.

Create a Kubernetes Secret

With your existing secret-management workflow, create a Kubernetes Secret named langsmith-forge-github with these keys:

Key	Value
`forge_github_client_secret`	GitHub client secret
`forge_github_state_jwt_secret`	Separately generated state JWT secret
`forge_github_app_pem`	Contents of the GitHub App private-key PEM
`forge_github_webhook_secret`	Webhook secret also configured in GitHub

Do not put these values in Helm values or command-line arguments. For production deployments, use your existing secrets workflow, such as Sealed Secrets or External Secrets Operator. See Use an existing secret for more.

Add the configuration to your langsmith_config.yaml

Add the following to hostBackend.deployment.extraEnv in your langsmith_config.yaml. Reference the sensitive values with secretKeyRef; never set them through commonEnv or as inline values:

hostBackend:
  deployment:
    extraEnv:
      - name: FORGE_GITHUB_APP_ID
        value: "<app-id>"
      - name: FORGE_GITHUB_APP_PUBLIC_LINK
        value: "https://github.com/apps/<app-slug>"
      - name: FORGE_GITHUB_CLIENT_ID
        value: "<client-id>"
      - name: FORGE_GITHUB_CLIENT_SECRET
        valueFrom:
          secretKeyRef:
            name: langsmith-forge-github
            key: forge_github_client_secret
      - name: FORGE_GITHUB_STATE_JWT_SECRET
        valueFrom:
          secretKeyRef:
            name: langsmith-forge-github
            key: forge_github_state_jwt_secret
      - name: FORGE_GITHUB_APP_PEM
        valueFrom:
          secretKeyRef:
            name: langsmith-forge-github
            key: forge_github_app_pem
      - name: FORGE_GITHUB_WEBHOOK_SECRET
        valueFrom:
          secretKeyRef:
            name: langsmith-forge-github
            key: forge_github_webhook_secret

Then apply the updated chart:

helm upgrade -i langsmith langchain/langsmith --values langsmith_config.yaml --version <version> -n <namespace> --wait --debug

Install the app on repositories

Once pods are healthy, install the GitHub App on the repositories Engine should access:

Open the app’s public link (FORGE_GITHUB_APP_PUBLIC_LINK) and click Install, or open Settings > Applications > GitHub Apps in your GitHub organization.
Select the repositories Engine should access. If the installation does not grant access to all repositories, explicitly select each private repository Engine needs.
In LangSmith, open a tracing project, go to the Engine tab, and select the repository in the GitHub Repository field.

​LangSmith Cloud

​Self-hosted

​Create a GitHub App

​See also

LangSmith Cloud

Self-hosted

Create a GitHub App

See also