ScrapingBee icon

ScrapingBee

Scrape websites using ScrapingBee

Actions16

  • Amazon Pricing API Actions
  • Amazon Product API Actions
  • Amazon Search API Actions
  • ChatGPT API Actions
  • Fast Search API Actions
  • Gemini API Actions
  • Google Search API Actions
  • HTML API Actions
  • Usage Actions
  • Walmart Product API Actions
  • Walmart Search API Actions
  • YouTube Metadata API Actions
  • YouTube Search API Actions
  • YouTube Subtitles API Actions

HTML API → POST

AI-generated

Summary

Send a POST request to a specified webpage using the ScrapingBee HTML API, optionally forwarding headers and including a customizable request body.

Inputs

  • URL (required) — The target webpage URL to send the POST request to.
  • Forward Headers — Toggle to forward ScrapingBee and custom headers to the target webpage.
  • Specify Headers As — Choose how to provide forwarded headers, either as key/value pairs or raw JSON. Only applicable if Forward Headers is true.
  • Headers (JSON) — Set forwarded headers in raw JSON format, requiring keys prefixed with 'Spb-'. Only when Forward Headers is true and header spec is JSON.
  • Headers (Key/Value) — Set forwarded headers as individual key/value pairs without prefix. Only when Forward Headers is true and header spec is keypair.
  • Send Body — Toggle to include a request body in the POST.
  • Body Content Type — Specify the Content-Type of the request body: form-url-encoded, JSON, or raw text. Only applies if Send Body is true.
  • Body Parameters — Set body parameters as key/value pairs for form-urlencoded content type. Visible only when Body Content Type is form-url-encoded and Send Body is true.
  • Body (JSON) — Set the raw JSON body content. Visible only when Body Content Type is JSON and Send Body is true.
  • Body (Raw) — Set the raw request body as plain text. Visible only when Body Content Type is raw and Send Body is true.
  • Content-Type (Raw) — Specify the Content-Type header for the raw body content. Only for Body Content Type = raw and Send Body true.
  • Additional Fields — Various optional parameters to customize the scraping request, such as blocking ads, rendering JavaScript, setting cookies, controlling device type, taking screenshots, AI extraction options, timeouts, proxies, and more.

Output shape

a list with one item per input, each containing the scraped response data either as parsed JSON, plain text, or binary content (e.g., screenshots) depending on the response's content type.

Responses with content-type application/json are parsed and returned as JSON objects. Text responses are returned as raw text. Other content types are returned as binary data with optional metadata like filename. Error responses return a JSON object with an error message. The node supports sending a POST method with optional body and headers through the ScrapingBee HTML API.

Links

Discussion