Callbacks

📄️ Argilla

Argilla is an open-source data curation platform for LLMs.

📄️ Comet Tracing

There are two ways to trace your LangChains executions with Comet:

📄️ Context

Context provides user analytics for LLM-powered products and features.

Fiddler is the pioneer in enterprise Generative and Predictive system ops, offering a unified platform that enables Data Science, MLOps, Risk, Compliance, Analytics, and other LOB teams to monitor, explain, analyze, and improve ML deployments at enterprise scale.

📄️ Infino

Infino is a scalable telemetry store designed for logs, metrics, and traces. Infino can function as a standalone observability solution or as the storage layer in your observability stack.

📄️ Label Studio

Label Studio is an open-source data labeling platform that provides LangChain with flexibility when it comes to labeling data for fine-tuning large language models (LLMs). It also enables the preparation of custom training data and the collection and evaluation of responses through human feedback.

📄️ LLMonitor

LLMonitor is an open-source observability platform that provides cost and usage analytics, user tracking, tracing and evaluation tools.

📄️ PromptLayer

PromptLayer is a platform for prompt engineering. It also helps with the LLM observability to visualize requests, version prompts, and track usage.

📄️ SageMaker Tracking

Amazon SageMaker is a fully managed service that is used to quickly and easily build, train and deploy machine learning (ML) models.

📄️ Streamlit

Streamlit is a faster way to build and share data apps.

📄️ Trubrics

Trubrics is an LLM user analytics platform that lets you collect, analyse and manage user

📄️ uptrain

UpTrain [github || website || docs] is an open-source platform to evaluate and improve LLM applications. It provides grades for 20+ preconfigured checks (covering language, code, embedding use cases), performs root cause analyses on instances of failure cases and provides guidance for resolving them.