Remove Duplicates
AI-generatedSummary
Remove duplicate records from the input JSON data based on specified unique fields or entire record content if no fields are specified.
Inputs
- Unique Fields — Comma-separated list of field names to identify duplicates. If empty, duplicates are identified by entire record content.
Output shape
a list of JSON records
Returns the input records filtered to exclude duplicates. Duplicate detection is done by concatenating specified unique field values or by JSON stringification of the whole record if no fields specified.