VLM Run icon

VLM Run

Interact with VLM Run API

Actions8

Chat Completion

AI-generated

Overview

This node performs chat completions using the VLM Run API, supporting various input types such as text, image URLs, video URLs, and file URLs. It allows users to send a series of messages with roles (assistant, system, user) and receive AI-generated chat responses. The node supports specifying the model to use, formatting the response with JSON schema, and optionally simplifying the output to just the message content. It is useful for generating conversational AI responses, processing multimedia inputs, and integrating structured response formats.

Use Case Examples

  1. Generating AI chat responses based on user prompts with text input.
  2. Including image, video, or file URLs in the chat prompt to enhance context for the AI.
  3. Parsing the AI response content as JSON for structured data extraction.
  4. Simplifying the output to only the message content for easier downstream processing.

Properties

Name Meaning
Model Model to use for chat completion, with options like vlmrun-orion-1:fast, vlmrun-orion-1:auto, vlmrun-orion-1:pro.
Prompt Collection of messages forming the chat prompt, each with a role (assistant, system, user) and content string.
Input Type Type of input provided to the chat completion, such as text, image URLs, video URLs, or file URLs.
Output Content as JSON Whether to attempt to parse the message content as a JSON object (only applicable for text input).
Image URL(s) Image URLs to include in the chat completion when input type is image.
Video URL(s) Video URLs to include in the chat completion when input type is video.
File URL(s) File URLs to include in the chat completion when input type is file (e.g., PDFs, documents).
Simplify Output Whether to return only the message content instead of the full API response.
Response Format Specify the format of the response using a JSON schema following JSON Schema Draft 7 specification, optionally with a name property for schema identification.

Output

JSON

  • choices - Array of chat completion choices returned by the API, each containing index, message (role, content, refusal, annotations), logprobs, and finish_reason.

Dependencies

  • VLM Run API
  • API key credential for authentication

Troubleshooting

  • Ensure at least one message is provided in the prompt; otherwise, an error is thrown.
  • Each message must have a 'role' property and non-empty content; empty or invalid content causes errors.
  • If using responseFormat, ensure it is a valid JSON object following the specified schema format to avoid parsing errors.
  • When inputType is not text, the last user message content is converted into an array including text and URLs of images, videos, or files.
  • If JSON output parsing is enabled, invalid JSON content in the response may cause parsing to fail.

Discussion