HasData icon

HasData

Scrape data from various websites

Web Scraping → Scrape Web Page

AI-generated

Summary

Scrape a web page and optionally extract structured data

Inputs

  • url — Web page address to scrape
  • additionalFields — Configurable extra fields for advanced scraping options
  • proxyType — Type of proxy to use (datacenter or residential)
  • proxyCountry — Country code for the proxy location
  • extractRules — JSON string defining CSS selectors for data extraction
  • screenshot — Capture a screenshot of the loaded page
  • extractEmails — Extract all email addresses found on the page
  • extractLinks — Extract all hyperlinks from the page
  • includeOnlyTags — CSS selector array to include only matching elements
  • excludeTags — CSS selector array to exclude matching elements
  • removeBase64Images — Remove base64‑encoded images from the response
  • aiExtractRules — JSON string for AI‑guided structured extraction with type hints
  • wait — Milliseconds to pause after page load
  • waitFor — CSS selector to wait for before proceeding
  • blockResources — Block loading of images, CSS, etc.
  • blockAds — Block advertisement content
  • blockUrls — List of URLs to block during fetch
  • jsScenario — Array of JavaScript actions to execute on the page
  • jsRendering — Enable full JavaScript rendering
  • headers — Custom HTTP headers to send with the request
  • outputFormat — Preferred response format(s) (html, text, markdown, json)

Output shape

object containing extracted data fields defined by extractRules or AI rules, plus optional metadata such as screenshots or status

Result shape varies with configuration; default is a JSON object with extracted fields, may include additional keys like screenshot or error

Examples

Example 1: Scrape a product page and extract title and price

url: https://example.com/product, extractRules: {"title": "h1", "price": ".price"}, outputFormat: json

Example 2: Capture a screenshot and extract all emails from a page

url: https://example.com, screenshot: true, extractEmails: true, outputFormat: text

Links

Discussion