Actions5
- Extractor Actions
- Scraping Agent Actions
- Reusable Scraper Actions
Scraping Agent → Interact and Extract
AI-generatedOverview
The AI Scraper node's 'Scraping Agent' resource with the 'Interact and Extract' operation allows users to navigate, interact with, and extract data from a webpage using an AI agent. This operation is useful for complex web scraping tasks where simple URL extraction is insufficient, such as interacting with dynamic content or extracting structured data based on a descriptive prompt. For example, it can be used to extract all product names and prices from an e-commerce page by providing a URL and a natural language prompt describing the desired data.
Use Case Examples
- Extract product details from a dynamic webpage by providing the URL and a prompt describing the data to extract.
- Use the AI agent to interact with a webpage that requires navigation or interaction before data extraction.
Properties
| Name | Meaning |
|---|---|
| URL | The URL of the webpage to extract data from. This is a required input for the operation. |
| Prompt | A natural language description of what data to extract from the page. This guides the AI agent in identifying and extracting the relevant information. This is required. |
| Define Output Schema | A boolean option to specify whether to define output columns to structure the extracted data. |
| Columns Input Mode | Select how to define columns for the output schema, either as separate fields or using a JSON schema. This is shown only if output schema definition is enabled. |
| Columns | Define the data fields (columns) to extract, including the name, type, and optional description for each column. This is shown only if output schema is enabled and columns input mode is 'fields'. |
| Schema (JSON) | Define columns as a JSON object schema, where each key is a field name and the value describes the data type and optional description. This is shown only if output schema is enabled and columns input mode is 'json'. |
Output
JSON
data- The extracted data from the webpage as structured JSON, based on the prompt and optional output schema.
Dependencies
- Parsera API (https://api.parsera.org) with an API key credential
Troubleshooting
- Ensure the provided URL is accessible and correct; inaccessible URLs will cause request failures.
- The prompt must clearly describe the data to extract; vague prompts may lead to incomplete or incorrect extraction.
- If defining output columns, ensure the schema matches the expected data structure to avoid parsing errors.
- Network issues or proxy misconfiguration can cause request timeouts or failures; verify proxy settings if used.
- SSL certificate issues can be bypassed by enabling the 'Ignore SSL Issues' option, but this may reduce security.
Links
- Parsera API Agent Extract Documentation - Official documentation for the AI agent extract endpoint used by this operation.