Actions16
- Marketplace Dataset Actions
- Web Scraper Actions
- Web Unlocker Actions
Marketplace Dataset → Deliver Snapshot
AI-generatedOverview
This node operation delivers a snapshot from the Bright Data Marketplace Dataset to a specified destination. It supports multiple delivery types including cloud storage services (Amazon S3, Aliyun OSS, Google Cloud Storage, Microsoft Azure), messaging services (Google Cloud PubSub), SFTP servers, Snowflake data warehouse, and webhooks. Users can specify the snapshot ID, delivery type, filename template, file extension, and various credentials and configuration options depending on the delivery method. This operation is useful for automating the transfer of dataset snapshots to preferred storage or processing endpoints for further analysis or integration.
Use Case Examples
- Deliver a dataset snapshot to an Amazon S3 bucket for storage and later processing.
- Send a snapshot to a Google Cloud PubSub topic to trigger downstream data processing workflows.
- Upload a snapshot file to an SFTP server for secure file transfer.
- Deliver a snapshot to a Snowflake stage for immediate querying and analysis.
Properties
| Name | Meaning |
|---|---|
| Snapshot ID | The ID of the snapshot to operate on, required to identify which snapshot to deliver. |
| Deliver Type | The destination type where the snapshot will be delivered, such as S3, Azure, webhook, etc. |
| Webhook Endpoint | The URL of the webhook endpoint to deliver the snapshot to, required if deliver type is webhook. |
| Filename Template | Template for the filename of the delivered snapshot file, including placeholders, required for all delivery types. |
| File Extension | The file extension for the delivered file, such as JSON, JSONL, or CSV, required for all delivery types. |
| Topic ID | Google PubSub topic ID, required if deliver type is Google Cloud PubSub. |
| Client Email | Client email for Google Cloud credentials, required for Google Cloud PubSub and Google Cloud Storage delivery types. |
| Private Key | Private key for Google Cloud credentials, required for Google Cloud PubSub and Google Cloud Storage delivery types. |
| Attributes | Attributes to include in the PubSub message, optional for Google Cloud PubSub delivery. |
| Container | Name of the Azure container, required for Microsoft Azure delivery. |
| Bucket | Name of the bucket for S3, Aliyun OSS, or Google Cloud Storage delivery, required for these delivery types. |
| AWS Access Key | AWS Access Key ID, required for Amazon S3 delivery. |
| AWS Secret Key | AWS Secret Access Key, required for Amazon S3 delivery. |
| Access Key | Access key for Aliyun OSS delivery. |
| Secret Key | Secret key for Aliyun OSS delivery. |
| Account | Azure storage account name, required for Microsoft Azure delivery. |
| Key | Azure storage key, required for Microsoft Azure delivery. |
| SAS Token | Azure SAS token for access, required for Microsoft Azure delivery. |
| Role ARN | AWS Role ARN for Amazon S3 delivery, optional. |
| External ID | External ID for AWS role assumption in Amazon S3 delivery, optional. |
| Directory | Target path or directory in the destination storage or server. |
| Region | AWS region for Amazon S3 or Aliyun OSS delivery. |
| Host | SFTP server host, required for SFTP delivery. |
| Port | SFTP server port, default is 22, used for SFTP delivery. |
| Path | Remote path on the SFTP server to store the file, used for SFTP delivery. |
| Username | SFTP username, required for SFTP delivery. |
| Password | SFTP password, required for SFTP delivery. |
| SSH Key | SSH key for SFTP authentication, required for SFTP delivery. |
| Passphrase | Passphrase for the SSH key, optional for SFTP delivery. |
| Database | Snowflake database name, required for Snowflake delivery. |
| Schema | Snowflake schema name, required for Snowflake delivery. |
| Stage | Snowflake stage name, required for Snowflake delivery. |
| Role | Snowflake role, required for Snowflake delivery. |
| Warehouse | Snowflake warehouse, required for Snowflake delivery. |
| Account (Snowflake) | Snowflake account, required for Snowflake delivery. |
| User | Snowflake user, required for Snowflake delivery. |
| Password (Snowflake) | Snowflake password, required for Snowflake delivery. |
| Compress | Boolean flag to indicate whether to compress the response in gzip format. |
Output
JSON
snapshotId- The ID of the snapshot that was delivered.deliveryStatus- Status of the delivery operation, indicating success or failure.deliveryDetails- Additional details about the delivery, such as destination info or error messages.
Dependencies
- Bright Data API key credential
Troubleshooting
- Ensure all required credentials for the chosen delivery type are provided and valid to avoid authentication errors.
- Verify that the snapshot ID is correct and exists in the Bright Data Marketplace to prevent not found errors.
- Check network connectivity and endpoint URLs, especially for webhook and SFTP delivery types, to avoid connection failures.
- For cloud storage deliveries, ensure the specified bucket/container and permissions are correctly configured.
- If using compression, verify that the receiving endpoint supports gzip format to prevent data handling issues.
Links
- Bright Data Marketplace Snapshot Delivery Documentation - Official documentation for delivering snapshots from Bright Data Marketplace.