Wiro - Rag Chat icon

Wiro - Rag Chat

Wiro LLM Conversational RAG Tool – A unified interface for seamless RAG-based conversations. Se

Wiro - Rag Chat

AI-generated

Overview

This node integrates with the Wiro LLM Conversational RAG (Retrieval-Augmented Generation) API to facilitate advanced conversational AI interactions. It allows users to send prompts along with multiple document URLs for context, and supports various parameters to customize the model's behavior, such as temperature, token limits, chunk size, and sampling options. This node is useful for scenarios where users want to leverage a powerful language model to generate responses based on both the prompt and relevant external documents, enabling applications like chatbots, document-based Q&A, and interactive AI assistants.

Use Case Examples

  1. A user uploads multiple documents and sends a prompt to get a context-aware answer from the model.
  2. A chatbot uses session and user IDs to maintain conversation history and provide personalized responses.
  3. An application adjusts temperature and sampling parameters to generate creative or deterministic text outputs.

Properties

Name Meaning
Select Model The language model to use for generating responses.
Select Model Private A private model option for generating responses.
Input Document Multiple URLs Multiple document files uploaded for providing context to the model. Supported file types include .csv, .docx, .epub, .jpeg, .jpg, .mbox, .md, .mp3, .mp4, .pdf, .png, .ppt, .pptm, .pptx.
Input Document Url Multiple Multiple file URLs separated by commas to provide context documents to the model. Supported file types are the same as above.
Prompt The main prompt text to send to the model for generating a response.
User ID A unique identifier for the user to store and retrieve chat history specific to that user.
Session ID A unique identifier for a session to manage multiple sessions for the same user and maintain separate chat histories.
System Prompt A system-level prompt prepended to the user prompt to guide the model's behavior.
Temperature Controls randomness of the output; 0 is deterministic, values above 1 increase randomness.
Top P Samples from the top p percentage of most likely tokens during text decoding.
Top K Samples from the top k most likely tokens during text decoding.
Chunk Size Size in tokens of each text chunk processed during retrieval phase.
Chunk Overlap Number of tokens overlapping between chunks to ensure context continuity.
Similarity Top K Number of most similar documents or chunks to retrieve based on similarity scores.
Context Window Maximum number of tokens the model can process at once, including input and retrieved chunks.
Max New Tokens Maximum number of tokens the model can generate in response.
Seed Seed value for randomization control.
Quantization Enables quantization to reduce model precision for lower memory usage and faster inference, at potential slight quality cost.
Do Sample Controls whether the model generates text deterministically or with randomness.

Output

JSON

  • taskid - Unique identifier for the processing task.
  • message - The generated response message from the model.
  • status - Status of the task, e.g., 'completed' or 'failed'.

Dependencies

  • Wiro API key and secret credentials for authentication

Troubleshooting

  • If the API response does not include a valid task ID or socket access token, the node throws an error indicating an invalid response from the Wiro API. Ensure API credentials are correct and the API service is reachable.
  • Polling for task completion may fail or timeout; verify network connectivity and API service status.
  • Incorrect or unsupported document URLs or file types may cause processing errors; ensure documents are accessible and properly formatted.

Links

  • Wiro RAG Chat API - Official API endpoint used by the node for conversational RAG processing.

Discussion