Parquet icon

Parquet

Read and write Apache Parquet files with multi-cloud storage support. Created by Gerd First-Grüttner.

Actions4

Write

AI-generated

Summary

Write JSON data items to an Apache Parquet file stored locally or in cloud storage (Azure Blob, AWS S3, Google Cloud Storage).

Inputs

  • Storage Type (required) — Select where to store the Parquet file: Local File, Azure Blob Storage, AWS S3, or Google Cloud Storage.
  • File Path / Container / Bucket and Blob / Key (required) — Specify the exact path or container/bucket and file name where the Parquet file will be written, depending on the storage type chosen.
  • Schema Mode (required) — Choose between automatic schema detection from input data or manually defining the schema in JSON format.
  • Manual Schema — Provide a JSON schema definition for the Parquet file when using manual schema mode.
  • Options — Configure write options such as compression algorithm (SNAPPY, GZIP, UNCOMPRESSED), strict schema validation to enforce data conformity, nested object handling method (native struct or JSON), row group size, and whether to enable progress logging during writing.

Output shape

a single JSON object with fields: success (boolean), rowsWritten (number), storageType (string), and compression (string).

Throws errors if no input data is provided or schema validation fails. Writes the Parquet file atomically to the specified storage location. Supports nested data structures according to options set.

Examples

Example 1: Write a set of JSON records to a local Parquet file using auto-detected schema and default Snappy compression.

Set Storage Type to 'Local File', specify File Path (e.g. /path/to/output.parquet), Schema Mode to 'Auto Detect', and provide input JSON data as items.

Links

Discussion