FireCrawl crawls and converts any website into LLM-ready data. It crawls all accessible subpages and give you clean markdown and metadata for each. No sitemap required.

​
Installation and Setup

Install the python SDK: 
pip install firecrawl-py==0.0.20

​
Document loader

See a usage example. 
from langchain_community.document_loaders import FireCrawlLoader