Olostep Web Scraper icon

Olostep Web Scraper

Search, extract, and structure web data using Olostep's Web search, scraping, and crawling API

Actions6

Scrape → Scrape Website

AI-generated

Summary

Scrape a single web page's content by specifying its URL and receive the extracted data in the chosen format.

Inputs

  • URL to Scrape (required) — The full URL (including http:// or https://) of the website page to extract content from.
  • Output Format — The desired format for the scraped output content; options are HTML, Markdown, JSON, or Plain Text (default is Markdown).
  • Country Code — An optional country code (e.g., US, GB, CA) to influence location-specific scraping behavior.
  • Wait Before Scraping (Milliseconds) — Milliseconds to wait before scraping the page, useful for pages with dynamic content (default is 0).
  • Parser — Optional parser ID string to enable specialized content extraction for certain types of pages (e.g., '@olostep/amazon-product').

Output shape

A single JSON object per input URL containing metadata and the scraped content in the requested format.

Returned data includes the scraped content fields (markdown_content, html_content, json_content, text_content) depending on the selected output format, status, timestamps, optional country and parser info, as well as hosted URLs for screenshots and scraped content previews if available.

Examples

Example 1: Scrape https://example\.com and get content in Markdown format, waiting 500ms before scraping.

URL to Scrape: https://example.com, Output Format: Markdown, Wait Before Scraping: 500

Discussion