n8n-nodes-spicrawl

This is an n8n community node. It lets you use Spicrawl in your n8n workflows.

Spicrawl is a web scraping API. It fetches any URL and returns it as clean markdown, HTML, text, a PDF or structured JSON. It renders JavaScript when a page needs it, and runs large URL lists as batch jobs.

n8n is a fair-code licensed workflow automation platform.

Installation
Operations
Credentials
Usage
Examples
Troubleshooting
Compatibility
Resources
Version history

Installation

Follow the installation guide in the n8n community nodes documentation, and install the package n8n-nodes-spicrawl.

Operations

The Spicrawl node has two operations.

The node is marked as usable as a tool, so an AI Agent node can call both operations.

Rendering tiers

Every page is loaded by the cheapest tier that can do what you asked for.

Rendering: Auto (Scrape URL only) starts with fetch and escalates to js only if the page needs it. You pay for the tier that worked. See Credits for current prices. A request that fails costs nothing.

Scrape URL parameters

Options:

Batch Scrape parameters

Options: Block Resources, Concurrency, Credit Budget, Custom Headers, Failure Threshold, Job Name, Main Content Only, Max Attempts, Max Cost Per Page (Credits), Max Wait (Minutes), Poll Interval (Seconds), Proxy URL, Wait After Load (Ms), Wait For Selector.

Batch jobs run each URL as one fetch or one render. Extraction, screenshots, PDFs, actions, Auto rendering and the result cache are not available in a batch; the API refuses them with a 400 that names each offending field, and nothing is charged. Use Scrape URL for those.

Batch Scrape runs once for all input items, not once per item. The job is polled until it finishes (or Max Wait is reached), then every finished page is returned.

Credentials

You need a Spicrawl account and an API key.

1. Sign in at app.spicrawl.com and open API Keys.
2. Create a key. Live keys start with spicrawl_live_ and spend credits. Test keys start with spicrawl_test_.
3. Give the key the scrape scope for Scrape URL and the batch scope for Batch Scrape.
4. In n8n, create a Spicrawl API credential and paste the key into API Key.

n8n tests the credential by listing your recent requests, which works with any valid key. See Authentication.

Usage

Output of Scrape URL

With Simplify on you get these fields, when they apply:

Turn Simplify off to also get engine (the tier that served the page), credits_remaining, cache_state, proxy_source and the rest of the JSON envelope. PDFs and screenshots always arrive as binary data, whatever Simplify is set to.

Output of Batch Scrape

One item per finished URL, in submission order:

Failed URLs are returned too, so filter on status with an IF or Filter node. If nothing finished, the node returns the job itself so you can see why.

Batch results stay readable for 72 hours. If the job does not finish within Max Wait, the node stops with an error that names the job ID; the job keeps running and finished pages are still charged.

Notes

• Scraping is not idempotent: every call is charged, including cache hits. Use n8n's Retry On Fail for errors marked retryable, not for successful calls.
• Errors from the API are shown with their stable code, for example ERR::AUTH::INSUFFICIENT_SCOPE, plus the API's hint and the request ID.
• Use the markdown format when the output goes to an AI model: it is much smaller than HTML.

Examples

Import an example with Workflow menu > Import from URL or clipboard. Select your credential on the Spicrawl node afterwards.

Scrape a page to markdown

{
  "nodes": [
    {
      "name": "When clicking Test workflow",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [0, 0],
      "parameters": {}
    },
    {
      "name": "Spicrawl",
      "type": "n8n-nodes-spicrawl.spicrawl",
      "typeVersion": 1,
      "position": [220, 0],
      "parameters": {
        "operation": "scrape",
        "url": "https://example.com",
        "responseFormat": "markdown",
        "rendering": "fetch",
        "simplify": true,
        "options": {}
      }
    }
  ],
  "connections": {
    "When clicking Test workflow": {
      "main": [[{ "node": "Spicrawl", "type": "main", "index": 0 }]]
    }
  }
}

Extract product fields from a JavaScript page

Set Rendering to JavaScript, then add the options Wait For Selector (.price) and Extract Fields (JSON):

{
  "title": "h1",
  "price": ".price",
  "images": { "selector": ".gallery img", "kind": "list", "output": "attr", "attribute": "src" },
  "next_page": "a.next@href"
}

The fields come back under data:

{
  "url": "https://example.com/products/42",
  "status": 200,
  "credits": 3,
  "data": { "title": "Walnut Desk", "price": "$349.00", "images": ["https://example.com/img/desk-1.jpg"] },
  "empty_fields": ["next_page"]
}

Take a screenshot

Add the options Screenshot and Screenshot Full Page. The image arrives in the binary property screenshot, ready for a Write Binary File, Google Drive or Send Email node. The call uses the browser tier.

Batch scrape a list of URLs

{
  "nodes": [
    {
      "name": "When clicking Test workflow",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [0, 0],
      "parameters": {}
    },
    {
      "name": "Spicrawl",
      "type": "n8n-nodes-spicrawl.spicrawl",
      "typeVersion": 1,
      "position": [220, 0],
      "parameters": {
        "operation": "batch",
        "urlSource": "list",
        "urls": "https://example.com/products/1\nhttps://example.com/products/2",
        "responseFormat": "markdown",
        "rendering": "js",
        "simplify": true,
        "batchOptions": { "jobName": "nightly-catalog", "concurrency": 20 }
      }
    }
  ],
  "connections": {
    "When clicking Test workflow": {
      "main": [[{ "node": "Spicrawl", "type": "main", "index": 0 }]]
    }
  }
}

To scrape URLs that come from a previous node (a Google Sheet, an RSS feed, a database), set URL Source to Input Items and point URL at the field, for example {{ $json.link }}.

Give an AI agent a web page reader

Add a Spicrawl node to an AI Agent node's tools. The agent fills in the URL and picks the options it needs. Keep Response Format on Markdown, and set Max Cost (Credits) so one tool call can never spend more than you intend.

Troubleshooting

The full list is in the error catalogue.

Compatibility

Developed against n8n 2.x with n8n-workflow 2.42 (n8nNodesApiVersion 1). The package has no runtime dependencies.

Resources

• Source code, in the agent-plugins repository
• n8n community nodes documentation
• Spicrawl documentation
• Scrape API reference
• Batch jobs guide
• Structured data extraction guide

Version history

See CHANGELOG.md.

• 0.1.0: Scrape URL and Batch Scrape.

License

MIT