Stagehand icon

Stagehand

Control browser using Stagehand with CDP URL

Actions3

Extract

AI-generated

Overview

This node, named "Stagehand," enables controlling a browser session via the Chrome DevTools Protocol (CDP) URL and performing three main operations: acting on the page using natural language instructions, extracting structured data from the page, and observing the page to plan an action. It leverages an AI language model to interpret instructions and interact with the browser accordingly.

The Extract operation specifically allows users to extract structured data from a web page by providing an instruction and a schema describing the expected output format. The schema can be defined in multiple ways, including a field list, example JSON, JSON Schema, or manual Zod schema code. This flexibility makes it useful for scenarios such as scraping specific data points from dynamic web pages, automating data collection workflows, or integrating web content extraction into larger automation pipelines.

Practical examples:

  • Extracting product details like title, price, and availability from an e-commerce page.
  • Scraping article metadata such as headline, author, and publish date from news websites.
  • Collecting form input values or user comments from interactive web applications.

Properties

Name Meaning
CDP URL Chrome DevTools Protocol WebSocket URL to connect to the target browser instance (e.g., ws://localhost:9222/devtools/browser/...). Required to establish control over the browser session.
Instruction Natural language instruction describing what data to extract or what action to perform on the page.
Schema Source Method used to define the output data schema for extraction. Options:
- Field List
- Example JSON
- JSON Schema
- Manual Zod (custom Zod schema code)
Fields (Shown if Schema Source is "Field List") A list of fields specifying property names, their types (Array, Boolean, Number, Object, String), and whether they are optional in the extracted object.
Example JSON (Shown if Schema Source is "Example JSON") A sample JSON object illustrating the expected structure of the extracted data.
JSON Schema (Shown if Schema Source is "JSON Schema") A JSON Schema draft-07 document defining the expected structure and validation rules for the extracted data.
Zod Code (Shown if Schema Source is "Manual Zod") Custom Zod schema code written manually to describe the expected data structure.
Advanced Options Collection of additional options:
- Enable Caching: Whether to cache Stagehand operations (default true).
- Log Messages: Include internal Stagehand log messages in output.
- Verbose Level: Logging verbosity (No Logs, Only Errors, All Logs).

Output

The node outputs an array of items, each containing a JSON object with the following structure:

  • operation: The operation performed, here always "extract".
  • result: The extracted data object matching the provided schema.
  • messages (optional): If logging is enabled, an array of internal Stagehand log messages related to the operation.

If binary data were involved (not applicable here), it would be summarized accordingly, but this node focuses on JSON structured data extraction.

In case of errors during execution, the output item will contain an error object describing the failure, alongside any collected log messages if enabled.

Dependencies

  • Requires a running browser instance accessible via a Chrome DevTools Protocol WebSocket URL.
  • Needs an AI language model connection input that provides:
    • A model identifier string.
    • An API key credential for authentication.
  • Uses external libraries:
    • @browserbasehq/stagehand for browser control and AI-driven interaction.
    • zod for schema validation.
    • json-to-zod and json-schema-to-zod for converting JSON examples or schemas into Zod schemas.
  • No internal credential names are exposed; users must provide appropriate API keys and CDP URLs configured externally.

Troubleshooting

  • Common issues:

    • Invalid or unreachable CDP URL: Ensure the browser is running with remote debugging enabled and the URL is correct.
    • Missing or invalid AI model or API key: Confirm the AI language model input connection is properly configured with valid credentials.
    • Schema parsing errors: When using JSON Schema, Example JSON, or Manual Zod, syntax errors or incompatible schema definitions may cause failures.
    • Unsupported schema source: Only the four listed schema sources are supported; others will throw errors.
    • Operation not supported: Only "act", "extract", and "observe" operations are valid.

  • Error messages:

    • Error executing Stagehand operation: ... indicates a failure during the chosen operation, often due to invalid parameters or runtime issues.
    • Unsupported schema source: ... means the selected schema source is not recognized.
    • Unsupported operation: ... means the operation parameter is invalid.
    • Assertion errors about missing model or API key indicate misconfiguration of the AI language model input.

  • Resolutions:

    • Verify all required inputs are correctly set.
    • Validate JSON and Zod schema syntax before use.
    • Check connectivity to the browser and AI service.
    • Enable verbose logging to get detailed internal messages for debugging.

Links and References

Discussion