Apify icon

Apify

Access Apify tools for web scraping, data extraction, and automation.

Actions11

Actor → Scrape Single URL

AI-generated

Overview

This node interacts with the Apify platform to scrape a single URL using various crawler types. It is useful for extracting data from web pages by specifying the URL and choosing the appropriate crawler technology, such as Cheerio, JSDOM, or Playwright variants. This node supports authentication via API key or OAuth2, enabling secure access to Apify's scraping tools.

Use Case Examples

  1. Scrape product details from an e-commerce page by providing the product URL and selecting the Cheerio crawler for fast HTML parsing.
  2. Use the Playwright Firefox crawler to scrape dynamic content from a JavaScript-heavy website by specifying the URL and authenticating with an API key.

Properties

Name Meaning
URL The web address to be scraped. Must be a valid URL starting with http:// or https://.
Crawler Type The technology used to scrape the URL, affecting how the page is processed and rendered.
Authentication Method to authenticate with Apify's API, either via API key or OAuth2.

Output

JSON

  • data - The scraped data extracted from the specified URL.

Dependencies

  • Apify API service for web scraping and automation

Troubleshooting

  • Ensure the URL is valid and starts with http:// or https:// to avoid URL validation errors.
  • Verify that the selected crawler type is compatible with the target website's content and structure.
  • Check that the authentication credentials (API key or OAuth2 token) are correctly configured and have sufficient permissions.
  • Common errors may include network issues, invalid URLs, or authentication failures; reviewing error messages and logs can help diagnose these problems.

Links

Discussion