Anakin icon

Anakin

Scrape websites, search with AI, and extract structured data

Actions108

Web → Map

AI-generated

Summary

Discover all reachable URLs under a website starting from a specified URL, exploring link depth and breadth according to optional parameters.

Inputs

  • URL (required) — The starting URL for link discovery (typically a homepage or section root).
  • Additional Options — Collection of optional parameters including depth of link hops to follow, whether to include external links or subdomains, maximum number of results overall and per depth level, max wait time for job completion, polling interval, optional keyword filter to restrict returned URLs, and whether to use a headless browser rendering for SPAs.

Output shape

A JSON object containing a success flag, the operation type, job ID, the starting URL, and the mapping results including the list of discovered URLs.

The node submits an asynchronous mapping job to the Anakin Scraper API, polling until completion or timeout. Errors during execution, including timeouts or failures reported by the API, surface as errors in the node. The maximum wait time and polling intervals are configurable. The mapping includes URLs discovered up to a specified depth and breadth, with filters for external links, subdomains, and keyword matching.

Examples

Example 1: Map a website starting at its homepage to discover up to 50 URLs reachable within 2 clicks, excluding external links and subdomains.

Set the URL to the site's homepage. Use default or specified depth=2, limit=50, includeExternalLinks=false, includeSubdomains=false, and default polling settings.

Links

Discussion