BrightData icon

BrightData

Interact with Bright Data to scrape websites or use existing datasets from the marketplace to generate adapted snapshots

Actions16

Web Scraper → Download Snapshot

AI-generated

Overview

This node operation downloads a snapshot from the Bright Data Web Scraper service. It supports downloading the entire snapshot or splitting it into parts for large datasets, allowing batch downloads. Users can specify the snapshot ID, output format (JSON or CSV), and whether to compress the response using gzip. This is useful for retrieving scraped data snapshots efficiently, especially when dealing with large volumes of data.

Use Case Examples

  1. Downloading a complete snapshot in JSON format for further processing.
  2. Downloading a large snapshot in parts with batch size control to manage memory and network usage.
  3. Downloading a snapshot in CSV format for use in spreadsheet applications.

Properties

Name Meaning
Split Into Parts Whether to split the snapshot into parts for batch downloading large snapshots.
Batch Size The number of records to download in each batch when splitting the snapshot into parts.
Part The part number of the snapshot to download when splitting into parts.
Snapshot ID The ID of the snapshot to download.
Format The format of the data to be returned, either JSON or CSV.
Compress Whether to compress the response in gzip format.

Output

JSON

  • data - The downloaded snapshot data in the specified format.

Dependencies

  • Bright Data API key credential

Troubleshooting

  • Ensure the Snapshot ID is correct and exists; otherwise, the download will fail.
  • If splitting into parts, verify that the batch size and part number are within valid ranges to avoid incomplete or failed downloads.
  • Check network connectivity and proxy settings if the request times out or fails.
  • If using compression, ensure the client can handle gzip compressed responses.

Links

Discussion