Actions14
- Account Actions
- Discovery Actions
- Extraction Actions
- Result Actions
- Scraper Creation Actions
- Scraper Run Actions
Discovery → Crawl Website URLs
AI-generatedSummary
Crawls a website to discover URLs, with controls for crawl depth, pages evaluated, result count, and URL pattern filtering.
Inputs
- URL (required) — Website URL to crawl.
- Max Depth — Maximum crawl depth; defaults to 2.
- Max Pages — Maximum pages evaluated; defaults to 50.
- Limit — Maximum number of results to return; defaults to 50.
- Include Patterns — Pipe-separated regular expressions for URLs to include; empty by default.
- Exclude Patterns — Pipe-separated regular expressions for URLs to exclude; empty by default.
- Request Options — Optional request settings: batching size and interval, whether to accept invalid SSL certificates, an HTTP proxy, and timeout. Defaults include batches of 50 with a 1000 ms interval, SSL verification enabled, and a 10000 ms timeout.
Output shape
Not determinable from the supplied source pack.
The operation module and node entry module contain only source map comments, so runtime request and return behavior are unavailable. The properties declare URL discovery and a maximum result count, but do not specify the returned data shape.