Olostep Web ScraperInstall

Summary

Extract all URLs from a specified website, optionally filtering and limiting the results based on search queries and URL patterns.

Inputs

• Website URL (required) — The URL of the website to extract links from; must include the scheme (http:// or https://).
• Search Query — Optional keyword or phrase to filter the extracted URLs.
• Top N URLs — Optional limit on the number of URLs to return.
• Include URL Patterns — Optional glob patterns to include only URLs matching these paths (e.g., "/blog/**").
• Exclude URL Patterns — Optional glob patterns to exclude URLs matching these paths (e.g., "/admin/**").

Output shape

a single record containing metadata about the created map, total URLs found, and an array of extracted URLs matching the specified criteria

The output JSON includes fields such as map_id, object type, original URL, total_urls count, and the list of URLs. If filters such as search query, top N, include or exclude URL patterns are set, they will affect the urls list returned. The urls array may be empty if no matches are found.