Scraper - UpRock CrawlerInstall

Summary

Execute various web scraping and research commands using the UpRock crawl network, including crawling URLs, fetching rendered Markdown and HTML content, fetching resource URIs, and researching web queries across multiple countries.

Inputs

• Command (required) — Select the UpRock command to run: 'Crawl URL and Fetch Content', 'Crawl URL', 'Fetch Resource URI', or 'Research Web Query'.
• URL (required) — Target URL to process (required for 'fetch' and 'crawl_fetch' commands).
• Method — HTTP method for the initial crawl or fetch. Options: 'Crawl Full Page' (JavaScript rendering), 'GET', 'POST', 'PUT'.
• Body — Request body for POST and PUT methods, provided as plain text.
• Country — Geographic location to execute the crawl from, influencing the device's regional view (not the page topic).
• Device Type — Device type to simulate during crawl: 'Automatic', 'Desktop', or 'Mobile'.
• Timeout Seconds — Timeout in seconds for the crawl or fetch operation; at least 60 seconds recommended for full-page crawls.
• Retries — Number of speculative retries; the first successful attempt is used.
• Resource URI (required) — crawl:// or sweep:// resource URI returned by a previous UpRock command (required for 'resource_fetch').
• Query (required) — Search query terms (required for 'web_research').
• Max Results — Maximum number of search results to return when performing web research.
• Number of Sources — Maximum number of countries to query for geographic diversity in web research.
• Suggested Countries — List of countries from which to perform the web research, controlling the searcher's geographic perspective.

Output shape

a list of items, each containing the result of the selected UpRock command execution

Output structure depends on the chosen command: 'fetch' returns crawl metadata plus Markdown and HTML content; 'crawl_fetch' returns crawl data; 'resource_fetch' returns content fetched from resource URIs; 'web_research' returns search query results with geographic diversity. Errors include detailed debug info if enabled.

Links

• n8n community nodes installation guide