OpenAI Chat Model with LiteLLM
AI-generatedOverview
This node integrates with OpenAI chat models enhanced by LiteLLM, allowing users to generate AI completions with advanced control and structured JSON metadata. It is designed for scenarios where detailed traceability, session management, and custom metadata attachment are important, such as complex AI workflows, multi-user environments, or debugging AI interactions.
Practical examples include:
- Running AI chains that require passing contextual metadata like project name, environment, or workflow identifiers.
- Managing user sessions and attributing requests to specific users for audit or personalization.
- Using advanced model options like reasoning effort levels and response formats (text or guaranteed valid JSON).
Properties
| Name | Meaning |
|---|---|
| Credential | An API key credential required to authenticate with the OpenAI-compatible service enhanced by LiteLLM. |
| JSON Metadata | A collection of optional metadata fields attached to the model request: - Custom Metadata (JSON): Arbitrary structured JSON data to provide additional context. - Session ID: Identifier for grouping traces and managing sessions. - User ID: Optional identifier for user attribution and traceability. |
| Model | The AI model to use for generating completions. Can be selected from a list or specified by ID. Models supported depend on the API endpoint and may include OpenAI or compatible models. |
| Options | Additional parameters controlling the generation behavior: - Base URL: Override the default API base URL. - Frequency Penalty: Penalizes repeated tokens. - Max Retries: Number of retry attempts. - Maximum Number of Tokens: Limits token count in output. - Presence Penalty: Encourages new topics. - Reasoning Effort: Controls reasoning depth (low, medium, high). - Response Format: Output format, either plain text or JSON. - Sampling Temperature: Controls randomness. - Timeout: Max request duration in milliseconds. - Top P: Controls diversity via nucleus sampling. |
| Notice | Informational messages shown based on configuration, e.g., reminders about including "json" in prompts when using JSON response format or limitations when overriding Base URL. |
Output
The node outputs an object under the response field containing the AI model's completion result. This includes the generated text or JSON content depending on the selected response format.
- If the response format is set to "text", the output will be a regular text string.
- If set to "json_object", the output guarantees a valid JSON structure generated by the model.
No binary data output is produced by this node.
Dependencies
- Requires an API key credential for authentication with an OpenAI-compatible service enhanced by LiteLLM.
- Supports overriding the API base URL to connect to non-OpenAI endpoints.
- Uses the
@langchain/openailibrary internally for interfacing with the AI models. - Requires n8n environment configured with appropriate credentials and network access to the AI service.
Troubleshooting
Common Issues:
- Incorrect or missing API credentials will cause authentication failures.
- Using outdated or unsupported model IDs may result in errors or no response.
- When using JSON response format, forgetting to include the word "json" in the prompt can lead to invalid JSON output.
- Overriding the Base URL with incompatible endpoints might disable some features like tools calling or JSON response format.
Error Messages:
- Authentication errors typically indicate invalid or missing API keys; verify credentials.
- Timeout errors suggest the request took longer than allowed; consider increasing the timeout setting.
- Parsing errors in JSON mode usually mean the model did not produce valid JSON; ensure prompt instructions include "json" and use supported models released after November 2023.