Spider icon

Spider

Crawl and scrape websites using the Spider API

Actions7

Scrape

AI-generated

Summary

Scrape content from a single web page specified by its URL, returning the extracted data in a selectable format.

Inputs

  • URL (required) — The exact web page URL to scrape content from.
  • Return Format (required) — The format to return the scraped content in; options include Markdown, HTML, Plain Text, or XML.
  • Additional Fields — Optional settings to control scraping behavior, including anti-bot bypass, proxy usage, headless browser rendering for JavaScript, readability mode to extract main article content, and whether to include page metadata.

Output shape

a single item containing the scraped page content

Returns one JSON object per input item with scraped content in the specified format. Optional metadata and content processing can be enabled via Additional Fields. Errors will appear in the item JSON if continue on fail is enabled.

Links

Discussion