Overview

This node integrates with Databricks hosted large language models (LLMs) for chat-based interactions. It allows users to select a Databricks model serving endpoint and configure options such as maximum tokens, temperature, top-p sampling, conversation ID, and whether to return execution trace for debugging. The node is useful for scenarios where users want to leverage Databricks LLMs for generating conversational AI responses, such as customer support bots, virtual assistants, or any application requiring natural language understanding and generation.

Use Case Examples

1. Using a Databricks hosted LLM to generate responses in a chatbot workflow.
2. Configuring temperature and max tokens to control the creativity and length of generated text.
3. Tracking conversations across multiple requests using a unique conversation ID.

Properties

Output

JSON

• Model - The configured Databricks chat model instance ready to generate chat completions.

Dependencies

• Databricks API credentials (host URL and token) are required for authentication and access to the model serving endpoints.

Troubleshooting

• Ensure that the Databricks credentials (host and token) are correctly configured and valid to avoid authentication errors.
• If the model endpoint name or URL is incorrect or does not exist, the node will fail to retrieve or use the model.
• Check that the maximum tokens value does not exceed the allowed limit (32768) to prevent request errors.
• If conversation tracking is used, ensure the conversation ID is unique and consistent across requests to maintain context.
• Enable 'Return Trace' only for debugging as it may increase response size and latency.

Links

• Databricks Model Serving Documentation - Official Databricks documentation on model serving endpoints and usage.