Parquet icon

Parquet

Read and write Apache Parquet files with multi-cloud storage support. Created by Gerd First-Grüttner.

Actions4

Read

AI-generated

Summary

Read rows from an Apache Parquet file stored locally, via URL, or in cloud storage (Azure Blob, AWS S3, Google Cloud Storage) with options to limit rows, select columns, and include metadata.

Inputs

  • Storage Type (required) — Select the storage location of the parquet file: Local File, URL, Azure Blob Storage, AWS S3, or Google Cloud Storage.
  • File Path (required) — Absolute path to the parquet file on local filesystem (required if Storage Type is 'Local File').
  • URL (required) — HTTP(S) URL to download the parquet file from (required if Storage Type is 'URL').
  • Container Name (required) — Azure Blob Storage container name (required if Storage Type is 'Azure Blob Storage').
  • Blob Name (required) — Path to the blob within the container (required if Storage Type is 'Azure Blob Storage').
  • Bucket Name (required) — AWS S3 bucket name (required if Storage Type is 'AWS S3').
  • File Key (required) — Object key (path) within the AWS S3 bucket (required if Storage Type is 'AWS S3').
  • Bucket Name (GCS) (required) — Google Cloud Storage bucket name (required if Storage Type is 'Google Cloud Storage').
  • File Path (GCS) (required) — Path to the parquet file within the Google Cloud Storage bucket (required if Storage Type is 'Google Cloud Storage').
  • Options.Batch Size — Number of rows to process at once for memory efficiency (default 1000).
  • Options.Max Rows — Maximum number of rows to read; 0 for all rows (default 0).
  • Options.Columns — Comma-separated list of columns to read; leave empty to read all columns.
  • Options.Include Metadata — Boolean to include parquet file metadata (schema, row count, compression codec) in the first output item.
  • Options.Enable Progress Logging — Boolean to enable console logging of reading progress for large files.

Output shape

a list of items where each item's JSON property is a record from the parquet file; optionally the first item includes file metadata under _parquetMetadata

Reads sequentially up to 'Max Rows' rows, applying column filtering if specified. File metadata fields include schema, row count, and compression codec. For cloud storage or URL, the file is downloaded temporarily for reading. Errors are thrown for read failures or invalid parameters.

Links

Discussion