n8n-nodes-scrapeeverything

An n8n community node for the Scrape Everything API. Public social data as clean JSON. It pulls public data from Instagram, TikTok, X, YouTube, Facebook, Telegram, Spotify, Pinterest and Twitch: profiles, posts, reels, videos, comments, transcripts, hashtags, playlists and more. One node and one API key cover all nine platforms. You don't need any platform logins.

n8n is a fair-code licensed workflow automation platform.

• Installation
• Credentials
• Operations
• Credits
• Usage
• Example workflow
• Compatibility
• Development
• Resources

Installation

Follow the community nodes installation guide in the n8n docs.

1. In n8n, go to Settings > Community Nodes.
2. Click Install.
3. Enter n8n-nodes-scrapeeverything and confirm.

After installation, search for Scrape Everything in the nodes panel. The node is also marked usableAsTool, so AI Agent nodes can use it as a tool.

Credentials

1. Create a free account at scrapeeverything.com. New accounts get 100 free credits.
2. Copy your API key from the dashboard.
3. In n8n, create a new Scrape Everything API credential and paste the key.

The credential sends Authorization: Bearer <key> with every request. When you save it, n8n tests the key against GET /v1/account, which returns your balance and never spends credits.

Operations

Pick a Resource (the platform), then an Operation (the action). Required parameters appear as fields. Optional ones (limit, sort, lang and so on) are under Additional Fields. Some endpoints accept one of several identifiers, such as a post URL or a shortcode. Those fields are shown together, with a hint that says which combinations work.

The full list, with endpoints and every parameter, is in docs/OPERATIONS.md.

The Custom → Custom Request operation calls any endpoint with a raw path (for example /v1/youtube/transcript) and a JSON body. Use it for endpoints that were added after this node version was published, and for GET /v1/account. The path must start with /v1/ and may only contain letters, digits, /, _ and -. Full URLs, //, :, @, \ and .. are rejected, and the request always goes to https://api.scrapeeverything.com, so your API key is never sent to another host.

Options

• Cache Days (0–14): accept a cached result up to N days old. A cache hit costs 0 credits. 0 always fetches live data.
• Include Response Metadata: return the full { success, data, meta } envelope instead of only data. meta holds credits_charged, credits_remaining, cached and request_id.

Credits

Scrape Everything bills in credits. Most calls cost 1–3 credits, and each operation's description in the node shows its cost.

• Successful calls are charged the listed cost.
• A definitive "not found / private / age-restricted" answer (HTTP 404) is charged, because the lookup was done.
• Platform blocks, timeouts and server errors (5xx) are refunded automatically.
• Validation errors (400) and insufficient balance (402) are never charged.
• Cache hits (cache_days) are free.

Free accounts are limited to 60 requests per minute and 5 concurrent requests. Paid accounts get higher limits. For large lists, use n8n's Loop Over Items node with a small batch size.

Usage

• Errors: the API returns { success: false, message, code, request_id }. n8n surfaces the message on the node. Common codes: INVALID_API_KEY (401), INSUFFICIENT_CREDITS (402), NOT_FOUND, PRIVATE_ACCOUNT, NO_TRANSCRIPT (404), RATE_LIMITED, CONCURRENCY_LIMIT (429) and TARGET_BLOCKED (502, refunded, retry later). Turn on Retry On Fail in the node settings for 429/502.
• IDs: send X, TikTok and Pinterest IDs as strings. They are larger than JavaScript numbers can hold exactly.
• Media links: video_url/play_url values are signed CDN links that expire, usually within a few days. Download them promptly if you need the files.
• Responsible use: the API only returns publicly available data. Respect the platforms' terms and local privacy law (GDPR/CCPA), and don't build profiles of private individuals.

Example workflow

examples/instagram-profile-to-sheet.json takes a list of Instagram usernames, fetches each profile (1 credit each) and appends username, followers, posts and verification status to a Google Sheet. To use it, import the file in n8n (Workflows → Import from File), then select your Scrape Everything and Google Sheets credentials.

Compatibility

• Requires n8n 1.x or later with community nodes enabled.
• Built with @n8n/node-cli (declarative style). The package has no runtime dependencies.

Development

npm install
npm run build        # regenerates operations from catalog/catalog-snapshot.json, then compiles
npm run lint         # n8n community node lint rules
npm run dev          # local n8n with the node loaded (needs Docker/Podman or --external-n8n)
npm run sync-catalog # refresh the catalog snapshot from GET https://api.scrapeeverything.com/v1/scrapers

Operations are generated at build time by scripts/generate-operations.mjs from the public catalog (GET /v1/scrapers). The script writes nodes/ScrapeEverything/generated/operations.ts, docs/OPERATIONS.md and the table above. Don't edit those by hand. At runtime the node never reads files or environment variables.

.npmrc sets ignore-scripts=true. The reason is isolated-vm, a transitive dev dependency through n8n-workflow, which tries a native build on some Node/OS setups. Building and linting this package never runs it.

Verification checklist (n8n Creator Portal)

• Package name n8n-nodes-*, n8n-community-node-package keyword, MIT license
• Scaffolded with and linted by @n8n/node-cli (npm run lint passes)
• No runtime dependencies; no environment variable or file access at runtime
• English-only UI strings and README; credential test request
• Public GitHub repo scrapeeverything/n8n-nodes-scrapeeverything matching the npm repository field
• Published from GitHub Actions with npm provenance (.github/workflows/publish.yml)
• npx @n8n/scan-community-package n8n-nodes-scrapeeverything passes after publishing
• Submitted at the n8n Creator Portal

Resources

• Scrape Everything: get an API key (100 free credits)
• API catalog (JSON)
• n8n community nodes documentation

About Scrape Everything

Scrape Everything is a REST API for public social media data. Send a username, link or ID and get clean, typed JSON from Instagram, TikTok, X, YouTube, Facebook, Telegram, Spotify, Pinterest and Twitch. Pay per successful request with prepaid credits that never expire. New accounts start with 100 free credits. https://scrapeeverything.com

License

MIT