Documentation Index
Fetch the complete documentation index at: https://docs.langchain.com/llms.txt
Use this file to discover all available pages before exploring further.
Tavily is a search engine built specifically for AI agents (LLMs), delivering real-time, accurate, and factual results at speed. Tavily offers an Extract endpoint that can be used to extract content from one or more URLs.
Overview
Integration details
| Returns artifact | Native async | Return data | Pricing |
|---|
| ❌ | ✅ | raw content and images | 1,000 free credits / month |
Setup
The integration lives in the langchain-tavily package.
pip install -qU langchain-tavily
Credentials
We also need to set our Tavily API key. You can get an API key by visiting this site and creating an account.
import getpass
import os
if not os.environ.get("TAVILY_API_KEY"):
os.environ["TAVILY_API_KEY"] = getpass.getpass("Tavily API key:\n")
Instantiation
The tool accepts various parameters during instantiation:
extract_depth (optional, str): The depth of the extraction, either “basic” or “advanced”. Default is “basic ”.
include_images (optional, bool): Whether to include images in the extraction. Default is False.
For a comprehensive overview of the available parameters, refer to the Tavily Extract API documentation
from langchain_tavily import TavilyExtract
tool = TavilyExtract(
extract_depth="basic",
include_images=False,
)
Invocation
The Tavily extract tool accepts the following arguments during invocation:
urls (required): A list of URLs to extract content from.
- Both
extract_depth and include_images can also be set during invocation
NOTE: The optional arguments are available for agents to dynamically set, if you set an argument during instantiation and then invoke the tool with a different value, the tool will use the value you passed during invocation.
tool.invoke({"urls": ["https://en.wikipedia.org/wiki/Lionel_Messi"]})
{'results': [{'url': 'https://en.wikipedia.org/wiki/Lionel_Messi',
'raw_content': 'Lionel Messi - Wikipedia\nJump to content\nMain menu\nMain menu\nmove to sidebar hide\nNavigation\n\nMain page\nContents\nCurrent events\nRandom article\nAbout Wikipedia\nContact us\n\nContribute\n\nHelp\nLearn to edit\nCommunity portal\nRecent changes\nUpload file\nSpecial pages\n\n \nSearch\nSearch\nAppearance\n\nDonate\nCreate account\nLog in\n\nPersonal tools\n\nDonate\nCreate account\nLog in\n\nPages for logged out editors learn more\n\nContributions\nTalk\n\nContents\nm... (truncated) ',
'images': []}],
'failed_results': [],
'response_time': 0.02}
We can also invoke the tool with a model-generated ToolCall, in which case a ToolMessage will be returned:
# This is usually generated by a model, but we'll create a tool call directly for demo purposes.
model_generated_tool_call = {
"args": {"urls": ["https://en.wikipedia.org/wiki/Lionel_Messi"]},
"id": "1",
"name": "tavily",
"type": "tool_call",
}
tool_msg = tool.invoke(model_generated_tool_call)
# The content is a JSON string of results
print(tool_msg.content[:400])
{"results": [{"url": "https://en.wikipedia.org/wiki/Lionel_Messi", "raw_content": "Lionel Messi - Wikipedia\nJump to content\nMain menu\nMain menu\nmove to sidebar hide\nNavigation\n\nMain page\nContents\nCurrent events\nRandom article\nAbout Wikipedia\nContact us\n\nContribute\n\nHelp\nLearn to edit\nCommunity portal\nRecent changes\nUpload file\nSpecial pages\n\n \nSearch\nSearch\nAppearance\n\
Use within an agent
We can use our tools directly with an agent executor by binding the tool to the agent. This gives the agent the ability to dynamically set the available arguments to the Tavily search tool.
if not os.environ.get("OPENAI_API_KEY"):
os.environ["OPENAI_API_KEY"] = getpass.getpass("OPENAI_API_KEY:\n")
from langchain.chat_models import init_chat_model
model = init_chat_model(model="gpt-5.4", model_provider="openai", temperature=0)
from langchain_tavily import TavilyExtract
from langchain.agents import create_agent
tavily_extract_tool = TavilyExtract()
agent = create_agent(model, [tavily_extract_tool])
user_input = "['https://en.wikipedia.org/wiki/Albert_Einstein','https://en.wikipedia.org/wiki/Theoretical_physics']"
for step in agent.stream(
{"messages": user_input},
stream_mode="values",
):
step["messages"][-1].pretty_print()
================================ Human Message =================================
['https://en.wikipedia.org/wiki/Albert_Einstein','https://en.wikipedia.org/wiki/Theoretical_physics']
================================== Ai Message ==================================
Tool Calls:
tavily_extract (call_BAK906Cpy8fDZttwqYTMdkKp)
Call ID: call_BAK906Cpy8fDZttwqYTMdkKp
Args:
urls: ['https://en.wikipedia.org/wiki/Albert_Einstein']
extract_depth: advanced
include_images: False
tavily_extract (call_4NFRB92QpiI5jnCTGr76dgMX)
Call ID: call_4NFRB92QpiI5jnCTGr76dgMX
Args:
urls: ['https://en.wikipedia.org/wiki/Theoretical_physics']
extract_depth: advanced
include_images: False
================================= Tool Message =================================
Name: tavily_extract
{"results": [{"url": "https://en.wikipedia.org/wiki/Theoretical_physics", "raw_content": "Published Time: 2003-09-21T15:11:11Z\nTheoretical physics - Wikipedia\nJump to content\nMain menu\nMain menu\nmove to sidebar hide\nNavigation\n\nMain page\nContents\nCurrent events\nRandom article\nAbout Wikipedia\nContact us\n\nContribute\n\nHelp\nLearn to edit\nCommunity portal\nRecent changes\nUpload file\nSpecial pages\n\n \nSearch\nSearch\nAppearance\n\nDonate\nCreate account\nLog in\n\nPersonal tools\n\nDonate\nCreate account\nLog in\n\nPages for logged out editors learn more\n\nContributions\nTalk\n\nContents\nmove to sidebar hide\n\n(Top)\n\n1 Overview\n\n\n2 History\n\n\n3 Mainstream theories... (truncated) ...", "images": []}], "failed_results": [], "response_time": 0.01}
================================== Ai Message ==================================
Here is a summary of the content from the Wikipedia pages on Albert Einstein and Theoretical Physics:
### Albert Einstein
Albert Einstein was a German-born theoretical physicist renowned for developing the theory of relativity, which revolutionized our understanding of space, time, and energy. His mass-energy equivalence formula, E=mc², is one of the most famous equations in physics. Einstein received the 1921 Nobel Prize in Physics for his explanation of the photoelectric effect, which was pivotal in establishing quantum theory.
Einstein's early life was marked by a deep interest in mathematics and physics, leading him to study at the Swiss Federal Polytechnic in Zurich. He worked at the Swiss Patent Office while developing his theories. His "miracle year" in 1905 saw the publication of four groundbreaking papers, including those on special relativity and the photoelectric effect.
Throughout his career, Einstein made significant contributions to statistical mechanics, quantum theory, and cosmology. He was a vocal advocate for civil rights and pacifism, and he played a role in alerting the U.S. to the potential of nuclear weapons during World War II. Einstein spent his later years at the Institute for Advanced Study in Princeton, New Jersey, where he continued to work on a unified field theory until his death in 1955.
### Theoretical Physics
Theoretical physics is a branch of physics that uses mathematical models and abstractions to explain and predict natural phenomena. It contrasts with experimental physics, which uses empirical methods to test these theories. Theoretical physics has been instrumental in developing major scientific theories, such as relativity and quantum mechanics.
The field involves creating models to describe physical events and is judged by how well these models predict empirical observations. Theoretical physics has a rich history, dating back to ancient philosophers and evolving significantly during the Scientific Revolution with figures like Galileo and Newton.
Modern theoretical physics seeks to unify existing theories and explore new concepts, often using advanced mathematics. It includes mainstream theories like the Big Bang, chaos theory, and quantum mechanics, as well as proposed and fringe theories that are still under investigation. Thought experiments, such as Schrödinger's cat, play a crucial role in theoretical physics by exploring scenarios that are not easily tested in reality.
Both pages highlight the interplay between theory and experimentation in advancing our understanding of the universe, with Einstein's work serving as a prime example of theoretical physics' impact.
API reference
For detailed documentation of all Tavily Extract API features and configurations head to the API reference: docs.tavily.com/documentation/api-reference/endpoint/extract