OmniParserInstall

Summary

Parse a UI screenshot image to detect interactive elements using AI vision by submitting a binary or base64-encoded image to the OmniParser API.

Inputs

• Input Type (required) — Select whether to provide the screenshot as binary data from previous node output or as a base64-encoded string.
• Binary Property (required) — Specify the name of the binary data property containing the image (required if Input Type is 'Binary Data').
• Base64 Image (required) — Provide the base64-encoded image string (required if Input Type is 'Base64 String').
• Box Threshold — Set the detection confidence threshold (0.0 to 1.0); lower values detect more elements.
• IOU Threshold — Set the Intersection over Union threshold (0.0 to 1.0) for merging overlapping detected boxes.
• Output Format (required) — Choose output format: a structured array of detected elements with coordinates, or the raw original API response.
• Include Annotated Image — Choose whether to include the base64 annotated image in the operation's output.

Output shape

a single item containing a JSON object

If 'Structured' output format is selected, the JSON includes 'elementsCount', an array of detected elements with their descriptions and coordinates, and optionally the annotated image as a base64 string. If 'Raw' is selected, the JSON includes the original API fields 'image' (optionally), 'parsed_content_list', and 'label_coordinates'. Errors are returned as JSON with an 'error' field if continue on fail is enabled.