Get Page Content
AI-generatedSummary
Crawls a starting URL with Crawl4AI and returns extracted page content. Choose a single page, same-domain links to depth 1, or a recursive same-domain crawl to depth 3, with optional controls for content filtering and browser behavior.
Inputs
- URL (required) — Starting HTTP or HTTPS URL to crawl.
- Crawl Scope — Single Page (default), Follow Links (same-domain links, depth 1), or Full Site (recursive same-domain crawl, depth 3).
- Options — Configure crawling and output: choose clean filtered markdown or complete raw markdown; optionally limit extraction with a CSS selector, include HTML or structured links, set browser profile and engine, headers, wait conditions and timeouts, cache behavior, ad/CSS blocking, and anti-bot or user simulation features. For multi-page crawls, set a page cap and exclude or deny URL patterns. A delay can wait for content to render before HTML is returned.
Output shape
A single record per input item, combining crawled pages when the scope is multi-page.
Returns page content as markdown, with crawl metadata and metrics; structured links are included by default, and raw HTML is included only when enabled. Clean markdown is the default; Complete prefers raw, unfiltered markdown. If crawling yields no pages, execution fails.