Firecrawl icon

Firecrawl

Scrape, crawl, map, search, and extract structured data from websites using Firecrawl API

Actions31

Scraping → Get batch scrape errors

AI-generated

Overview

This node operation retrieves error details for a specific batch scrape job from the Firecrawl API. It is useful for monitoring and debugging batch web scraping tasks by fetching any errors encountered during the batch scraping process. For example, if a user runs a batch scrape job to extract data from multiple web pages, this operation helps identify which pages or requests failed and why.

Use Case Examples

  1. A user runs a batch scrape job to collect product data from an e-commerce site and uses this operation to get error reports for failed requests in the batch.
  2. A developer automates monitoring of batch scraping jobs and triggers alerts based on errors retrieved using this operation.

Properties

Name Meaning
Batch ID The unique identifier of the batch scrape job for which to retrieve errors.
Use Custom Body Flag to indicate whether to use a custom request body instead of the default.
Request Options Additional HTTP request options such as batching, SSL certificate handling, proxy settings, and timeout configuration.

Output

JSON

  • errors - Array of error details for the batch scrape job.

Dependencies

  • Firecrawl API key credential

Troubleshooting

  • Ensure the Batch ID is correct and corresponds to an existing batch scrape job; otherwise, the API may return a not found error.
  • If using a proxy, verify the proxy URL and credentials are correct to avoid connection failures.
  • Timeout errors may occur if the server is slow or unresponsive; consider increasing the timeout setting.
  • If SSL certificate validation fails and the 'Ignore SSL Issues' option is not enabled, the request may be blocked.

Links

Discussion