BrightData icon

BrightData

Interact with Bright Data to scrape websites or use existing datasets from the marketplace to generate adapted snapshots

Actions14

Marketplace Dataset → Filter Dataset

AI-generated

Overview

This node interacts with the Bright Data platform to filter datasets from the Bright Data Marketplace. It allows users to apply various types of filters to datasets, such as single field filters, grouped filters using JSON, CSV filters, or JSON filters, and retrieve filtered data records. This is useful for users who want to extract specific subsets of data from large datasets based on custom criteria, for example, filtering a dataset to include only records where a field matches a certain value or a group of conditions are met.

Use Case Examples

  1. Filtering a dataset to include only records where the 'industry' field equals 'Accounting'.
  2. Applying multiple filters combined with 'and' logic to get records where 'age' is greater than 30 and 'name' equals 'John'.
  3. Using a CSV or JSON filter to specify multiple values for filtering a dataset.

Properties

Name Meaning
Dataset Select the dataset to filter from the Bright Data Marketplace.
Records Limit Maximum number of records to include in the filtered snapshot.
Filter Type Type of filter to apply: single filter or group of filters combined with 'and'.
Field Name Field name to filter on (used for single filter type).
Operator Operator to use for the filter (e.g., equals, greater than, includes).
Field Value Value to filter on (used for single filter type).
Filters Group JSON filter for multiple filters combined with 'and' logic (used for group filter type).
CSV Filter CSV formatted filter values (used for CSV filter type).
JSON Filter JSON formatted filter values (used for JSON filter type).

Output

JSON

  • records_limit - Maximum number of records included in the filtered snapshot.
  • dataset_id - Identifier of the dataset filtered.
  • filter - Filter criteria applied to the dataset, can be single or group filter.
  • items - Filtered dataset records returned from the API.

Dependencies

  • Requires an API key credential for Bright Data API access.

Troubleshooting

  • Common issues include missing or invalid Dataset ID, which will cause the node to throw an error indicating the Dataset ID is required.
  • Invalid JSON format in the Filters Group or JSON Filter input will cause parsing errors; ensure the JSON is correctly formatted.
  • If the API request fails, the node throws an error with the API response message; check network connectivity and API credentials.

Links

Discussion