ScraperAPIInstall

Summary

Create a ScraperAPI crawler job that starts from a URL, crawls matching links within a depth or credit limit, and streams scrape results to a callback webhook. The job can run once or on a schedule, and its creation response is returned for use in later workflow steps.

Inputs

• Start URL (required) — URL where the crawl begins.
• Max Depth or Crawl Budget (required) — Set a positive maximum link depth (start URL is depth 0) or a positive ScraperAPI credit budget; the operation errors if neither is set.
• Regular Expression for URLs (required) — Pattern used to select additional URLs to crawl; defaults to .*.
• Callback URL (required) — Webhook URL that receives results from successful and failed scrape attempts throughout the job.
• Optional Crawler Parameters — Configure whether the job is enabled (default true), an inclusion or exclusion URL pattern, and a schedule interval (default once) and name. API parameters can set autoparse, country code, desktop or mobile device, redirect following, header retention, output format, premium or ultra premium proxies, JavaScript rendering, 404 retries, and proxy session number.

Output shape

A single item per input item, containing the resource name and the HTTP response body, headers, status code, and status message.

The response is from the POST request that creates the job. The job's scrape results are sent separately to the configured callback URL. If Continue On Fail is enabled, errors are returned as an item with an error field.

Links

• n8n community nodes installation guide