> ## Documentation Index
> Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AirbyteLoader integration

> Integrate with the AirbyteLoader document loader using LangChain Python.

> [Airbyte](https://github.com/airbytehq/airbyte) is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.

This covers how to load any source from Airbyte into LangChain documents

## Installation

In order to use `AirbyteLoader` you need to install the `langchain-airbyte` integration package.

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
pip install -qU langchain-airbyte
```

Note: Currently, the `airbyte` library does not support Pydantic v2.
Please downgrade to Pydantic v1 to use this package.

Note: This package also currently requires Python 3.10+.

## Loading documents

By default, the `AirbyteLoader` will load any structured data from a stream and output yaml-formatted documents.

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_airbyte import AirbyteLoader

loader = AirbyteLoader(
    source="source-faker",
    stream="users",
    config={"count": 10},
)
docs = loader.load()
print(docs[0].page_content[:500])
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
\`\`\`yaml
academic_degree: PhD
address:
  city: Lauderdale Lakes
  country_code: FI
  postal_code: '75466'
  province: New Jersey
  state: Hawaii
  street_name: Stoneyford
  street_number: '1112'
age: 44
blood_type: "O\u2212"
created_at: '2004-04-02T13:05:27+00:00'
email: bread2099+1@outlook.com
gender: Fluid
height: '1.62'
id: 1
language: Belarusian
name: Moses
nationality: Dutch
occupation: Track Worker
telephone: 1-467-194-2318
title: M.Sc.Tech.
updated_at: '2024-02-27T16:41:01+00:00'
weight: 6
```

You can also specify a custom prompt template for formatting documents:

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_core.prompts import PromptTemplate

loader_templated = AirbyteLoader(
    source="source-faker",
    stream="users",
    config={"count": 10},
    template=PromptTemplate.from_template(
        "My name is {name} and I am {height} meters tall."
    ),
)
docs_templated = loader_templated.load()
print(docs_templated[0].page_content)
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
My name is Verdie and I am 1.73 meters tall.
```

## Lazy loading documents

One of the powerful features of `AirbyteLoader` is its ability to load large documents from upstream sources. When working with large datasets, the default `.load()` behavior can be slow and memory-intensive. To avoid this, you can use the `.lazy_load()` method to load documents in a more memory-efficient manner.

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
import time

loader = AirbyteLoader(
    source="source-faker",
    stream="users",
    config={"count": 3},
    template=PromptTemplate.from_template(
        "My name is {name} and I am {height} meters tall."
    ),
)

start_time = time.time()
my_iterator = loader.lazy_load()
print(
    f"Just calling lazy load is quick! This took {time.time() - start_time:.4f} seconds"
)
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
Just calling lazy load is quick! This took 0.0001 seconds
```

And you can iterate over documents as they're yielded:

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
for doc in my_iterator:
    print(doc.page_content)
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
My name is Andera and I am 1.91 meters tall.
My name is Jody and I am 1.85 meters tall.
My name is Zonia and I am 1.53 meters tall.
```

You can also lazy load documents in an async manner with `.alazy_load()`:

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
loader = AirbyteLoader(
    source="source-faker",
    stream="users",
    config={"count": 3},
    template=PromptTemplate.from_template(
        "My name is {name} and I am {height} meters tall."
    ),
)

my_async_iterator = loader.alazy_load()

async for doc in my_async_iterator:
    print(doc.page_content)
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
My name is Carmelina and I am 1.74 meters tall.
My name is Ali and I am 1.90 meters tall.
My name is Rochell and I am 1.83 meters tall.
```

## Configuration

`AirbyteLoader` can be configured with the following options:

* `source` (str, required): The name of the Airbyte source to load from.
* `stream` (str, required): The name of the stream to load from (Airbyte sources can return multiple streams)
* `config` (dict, required): The configuration for the Airbyte source
* `template` (PromptTemplate, optional): A custom prompt template for formatting documents
* `include_metadata` (bool, optional, default True): Whether to include all fields as metadata in the output documents

The majority of the configuration will be in `config`, and you can find the specific configuration options in the "Config field reference" for each source in the [Airbyte documentation](https://docs.airbyte.com/integrations/).

***

<div className="source-links">
  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>

  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/document_loaders/airbyte.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>
</div>