Sitemap API for fast website URL discovery
Find related URLs from a site before crawling, indexing, auditing, or building a knowledge base.
Find the important URLs inside a website before deciding what to crawl, index, audit, or migrate.
Key site URLs ready for crawling.
Sections and page types made visible.
Prioritize links before crawling.
Useful for
Crawl queues / SEO audits / Indexing
Map the site before extracting content
The Sitemap endpoint accepts a URL and returns discovered links. Use sitemap mode for known sitemap.xml sources or all mode when you need broader link discovery across pages.
Complete Discovery
Detect all related pages and links within a website. Prepare for thorough content extraction with comprehensive site mapping.
Rapid Mapping
Generate site structure maps quickly and efficiently. Save time when planning your content extraction strategy.
Organized Structure
Receive site links in a structured, hierarchical format. Easily understand website organization and content relationships.
Smart Filtering
Automatically filter out irrelevant links and duplicate content. Focus only on valuable, unique pages worth extracting.
Map the site before extracting content
Find related URLs from a site before crawling, indexing, auditing, or building a knowledge base.
Submit the root URL and choose sitemap or all discovery mode.
Review the returned link list and filter for important sections.
Pass selected URLs to Crawl API or DeepCrawl for content extraction.
Best for
Extract all URLs from a website with our powerful sitemap generator
Preparing a crawl queue for documentation, blogs, or knowledge bases.
Auditing website structure before migration or SEO analysis.
Finding important URLs before selectively calling Crawl API.
Sitemap API
Extract all URLs from a website with our powerful sitemap generator
FAQ
What discovery modes are available?
Sitemap mode processes links from sitemap.xml. All mode attempts broader link discovery from pages when the sitemap is missing or incomplete.
How many credits does Sitemap API use?
A Sitemap API request costs 1 credit.