Decodo icon

Decodo

Decodo's Web Scraping API

Actions5

Universal

AI-generated

Overview

This node integrates with Decodo's Web Scraping API to scrape web pages based on user-defined parameters. It supports scraping any target URL with options to enable or disable JavaScript rendering, specify geolocation for the scraping request, and choose whether to retrieve results in markdown format. This node is useful for extracting web content dynamically or statically from various websites, including scenarios where location-specific content is needed or JavaScript-rendered content must be captured.

Use Case Examples

  1. Scraping product details from an e-commerce site with JavaScript rendering enabled.
  2. Extracting search results from a webpage with geolocation set to 'United States' to get region-specific data.
  3. Retrieving static HTML content from a URL without JavaScript rendering and outputting results in markdown format.

Properties

Name Meaning
URL Target URL to scrape, required for specifying the webpage to extract data from.
JS Rendering Whether pages are allowed to render content dynamically using JavaScript, enabling scraping of dynamic content.
Geolocation The country name from which the scraping request will be made, allowing location-specific content scraping. The first letter should be capitalized, e.g., 'United States'.
Markdown Whether results are retrieved in markdown format, applicable only when JavaScript rendering is disabled.

Output

JSON

  • json - The scraped data returned from the Decodo Web Scraping API, containing the extracted content from the target URL.

Dependencies

  • Decodo Web Scraping API
  • An API key credential for authentication

Troubleshooting

  • Common issues include invalid or missing API credentials, resulting in authentication errors. Ensure the API key credential is correctly configured.
  • Incorrect or malformed URLs can cause scraping failures; verify the URL format before execution.
  • If JavaScript rendering is enabled but the page content does not load as expected, check if the target site uses advanced anti-scraping techniques or requires additional headers or cookies.
  • Errors related to geolocation may occur if the country name is misspelled or not capitalized properly; ensure correct formatting as specified.

Links

Discussion