Outscraper icon

Outscraper

Interact with Outscraper API

Actions52

AI Scraper → Scrape

AI-generated

Overview

The AI Scraper node in the Outscraper integration allows users to scrape data from a specified website URL based on a custom query schema. It is useful for extracting structured information such as contacts, social media links, and about information from web pages. Users can perform synchronous or asynchronous scraping requests and optionally provide a webhook URL to receive a callback once the scraping task is completed.

Use Case Examples

  1. Extract contact details and social media profiles from a company website by providing the URL and a query schema describing the desired data.
  2. Scrape product information from an e-commerce site by specifying the product page URL and a schema outlining the data fields to extract.

Properties

Name Meaning
Query The website URL to scrape data from, e.g., https://outscraper.com/.
Query Schema A description of what data should be parsed and extracted from the specified web page.
Async Request Whether to make the scraping request asynchronously, allowing the task to run in the background.
Webhook A callback URL where Outscraper will send a POST request once the scraping task is finished.
Request Options Additional settings for the request such as batching, SSL certificate validation, proxy usage, and timeout duration.

Output

JSON

  • data - The scraped data extracted from the specified website according to the query schema.

Dependencies

  • Outscraper API key credential

Troubleshooting

  • Ensure the provided website URL is correct and accessible to avoid request failures.
  • Verify the query schema accurately describes the data to be extracted to prevent empty or incorrect results.
  • If using asynchronous requests, ensure the webhook URL is correctly configured to receive callbacks.
  • Check proxy and SSL settings if requests fail due to network or security issues.
  • Timeout errors may occur if the server is slow to respond; consider increasing the timeout setting.

Links

Discussion