ThordataInstall

Summary

Scrape content from any specified URL by sending a request to Thordata's Universal Scrape API, optionally rendering JavaScript and choosing the output format as HTML or a PNG screenshot.

Inputs

• URL (required) — The full URL of the webpage to scrape. If the URL does not start with 'http://' or 'https://', 'https://' will be prepended automatically.
• Output Format — Select the format of the output: either the raw HTML content of the page or a PNG screenshot image.
• Render JavaScript — Toggle whether the webpage's JavaScript should be executed/rendered before scraping. Defaults to true.

Output shape

a list of JSON objects, each representing the scraped page content

If 'Output Format' is 'HTML', the output JSON will contain the parsed JSON response or raw HTML string under the 'json' property. If 'Output Format' is 'PNG', the node returns a JSON message with a binary PNG screenshot file attached in the 'binary' property named 'data'. Errors are returned as JSON objects with an 'error' field if 'continueOnFail' is enabled.

Examples

Example 1: Scrape HTML content of a webpage with JavaScript rendered

Set the URL to the target webpage, leave Output Format as HTML, and ensure Render JavaScript is enabled.

Example 2: Capture a screenshot of a webpage without running JavaScript

Set the URL, change Output Format to PNG, and disable Render JavaScript.