http-to-s3

n8n node that streams an HTTP response body directly into an S3 bucket (no binary data held in n8n)

Package Information

Released: this month
Downloads: 0 weekly / 0 monthly
Latest Version: 1.0.0
Author: Suneet Kumar

Available Nodes

Documentation generating…

HTTP to S3 icon
HTTP to S3
Make an HTTP request and stream the response body straight into an S3 bucket, without holding it in n8n

Documentation

n8n-nodes-http-to-s3

This is an n8n community node. It makes an HTTP request and streams the response body directly into an
S3 bucket, like an HTTP Request node and an S3 upload node fused into one step.

n8n's binary data storage is the usual bottleneck for large downloads: in database mode a single file
is capped at 1024 MiB, and in filesystem mode the whole file lands on a volume first. This node never
puts the file into n8n at all. Bytes flow from the source server into an S3 multipart upload, and the
node outputs only JSON metadata about the stored object.

n8n is a fair-code licensed workflow automation platform.

Installation
Operations
Credentials
Usage
Compatibility
Resources
Version history

Installation

Follow the installation guide in the
n8n community nodes documentation and install n8n-nodes-http-to-s3.

This node depends on the AWS SDK, so it is not eligible for n8n Cloud verification and is meant for
self-hosted n8n.

With the n8n Helm chart, list it under
nodes.external.packages:

nodes:
  external:
    packages:
      - n8n-nodes-http-to-s3

Operations

The node has a single operation: request a URL and upload the response body to S3.

Parameter Description
Method GET, POST, PUT, PATCH or DELETE.
URL The http or https URL to download from.
Query Parameters Name/value pairs appended to the URL.
Headers Name/value pairs sent with the request, for example Authorization.
Body Raw request body for POST, PUT, PATCH and DELETE. Set a Content-Type header yourself if the server needs one.
Bucket Name The destination S3 bucket.
Object Key The destination key. Empty uses the last path segment of the final URL, after redirects.

Options:

Option Default Description
Concurrent Parts 4 Parts uploaded in parallel.
Content Type from server Content-Type stored on the S3 object.
Ignore SSL Issues off Download even if the server certificate cannot be verified (insecure).
Max Redirects 5 Redirects to follow.
Part Size (MB) 64 Size of each multipart chunk, at least 5.
Storage Class Standard Standard, Standard-IA, Intelligent-Tiering or Glacier Instant Retrieval.
Timeout (Ms) 60000 Maximum idle time (no data received) before the download is aborted.

Output

One item per input item, with JSON only (no binary data):

{
  "bucket": "my-bucket",
  "key": "downloads/big-file.zip",
  "location": "https://my-bucket.s3.us-east-1.amazonaws.com/downloads/big-file.zip",
  "eTag": "\"3858f62230ac3c915f300c664312c11f-80\"",
  "bytes": 5368709120,
  "statusCode": 200,
  "contentType": "application/zip",
  "sourceUrl": "https://example.com/big-file.zip"
}

A non-2xx response fails the item with the status code and the start of the response body. With
Continue On Fail enabled the item carries an error message instead.

Credentials

Create an S3 API credential:

Field Description
Use Default AWS Credentials Use whatever the n8n process can already see (IAM role, IRSA, pod identity, AWS_* environment variables). The key fields below are hidden.
Access Key ID, Secret Access Key A key pair allowed to write to the bucket.
Session Token Only for temporary STS credentials.
Region Bucket region, for example eu-central-1.
Custom Endpoint For S3-compatible stores such as MinIO or Cloudflare R2. Leave empty for AWS. Path-style addressing is used when set.

The credential test calls ListBuckets. A key that is not allowed to list buckets still passes, because
uploading does not need that permission. The key needs s3:PutObject and s3:AbortMultipartUpload on the
target keys.

Usage

  • Memory. Each in-flight part is buffered, so memory is roughly Part Size × Concurrent Parts. The
    defaults use about 256 MB. Lower either one on small pods.
  • Maximum file size. S3 allows 10,000 parts, so the largest file is Part Size × 10,000. The
    default 64 MB parts allow files up to about 625 GB.
  • Timeouts. The node keeps going as long as data keeps arriving. Check your workflow's execution
    timeout (EXECUTIONS_TIMEOUT) for very long transfers.
  • Failures. If the download or a part fails, the multipart upload is aborted so no orphaned parts are
    billed.
  • Not an AI tool. The node is deliberately not usable as an AI agent tool, because an agent should
    not be able to make your n8n instance download arbitrary URLs.

Compatibility

Built against n8n-workflow 2.x and tested with Node.js 20 and later. Requires a self-hosted n8n.

Resources

Version history

See CHANGELOG.md.

Discussion