Crawl4aiInstall

Summary

Perform web crawling of one or more specified URLs with customizable browser, anti-detection, authentication, and advanced configurations, returning a structured JSON summary of the crawl per URL.

Inputs

• URL Configuration (required) — One or multiple URLs to crawl, each specified as a valid string URL.
• Browser Configuration — Settings for the browser instance including headless mode toggle, viewport dimensions, and custom user agent string.
• Anti-Detection Settings — Controls to enable undetected or stealth browsing modes and select a CAPTCHA bypass strategy (None, 2Captcha, Anti-Captcha, Custom).
• Authentication Configuration — Options to enable authentication with types such as Basic Auth, Form Auth, OAuth2, API Key, or Session Cookie, including respective credentials and login URL.
• Advanced Configuration — Advanced crawling parameters like max retries, request timeout, concurrency level, proxy usage, and debug logging.

Output shape

a JSON array with one element per input item, each containing crawl details per specified URL or an error object if crawling setup validation failed.

Each output item includes success status, the URL, crawl operation details, configuration parameters used (browser, anti-detection, authentication, advanced), and a timestamp. If an error occurs, the output contains an error message, success set to false, and a timestamp.