Overview

This node performs web scraping operations, including extracting content from a single web page. Specifically, the 'Extract single page' operation fetches the HTML content of a given URL and extracts the page title, all

headings, and paragraphs longer than 40 characters. This is useful for scenarios where you want to programmatically gather structured content from a specific webpage, such as summarizing articles, extracting key headings, or collecting textual data for further processing.

Use Case Examples

1. Extracting the main headings and paragraphs from a news article URL to summarize its content.
2. Scraping product details from an e-commerce page by extracting headings and descriptive paragraphs.
3. Gathering textual content from a blog post for sentiment analysis or content categorization.

Properties

Output

JSON

• url - The URL of the scraped page.
• title - The extracted title of the page.
  h1 - An array of extracted

headings from the page.

• paragraphs - An array of extracted paragraphs from the page, filtered to include only those longer than 40 characters.
• error - Error message if the scraping operation fails.
• type - Type of error if an error occurs during scraping.

Dependencies

• Uses the 'got' HTTP client library to fetch web page content.

Troubleshooting

• Common issues include network timeouts or unreachable URLs, which result in error messages indicating request failures.
• If the page structure changes or the content is dynamically loaded via JavaScript, the node may not extract the expected data.
• Errors related to invalid URLs or request timeouts can be resolved by verifying the URL and ensuring the target server is reachable.