OpenAI Batch icon

OpenAI Batch

Execute batch requests to OpenAI API

Actions2

Chat Completion

AI-generated

Summary

Execute chat completion requests via OpenAI Batch API (50% cheaper, slower) or directly (faster, standard pricing). Input items are converted to chat messages and processed per mode; results returned per item.

Inputs

  • processingMode (required) — Whether to use the batch API or send requests directly
  • model (required) — The model to use for chat completion
  • inputMode (required) — How to provide input messages: simple (prompt) or advanced (message array)
  • prompt — The prompt to send; use expressions like {{ $json.prompt }} to reference input data
  • systemPrompt — Optional system prompt to set the behavior of the assistant
  • messages — The messages for the chat completion; each message has a role and content
  • options.maxTokens — Maximum number of tokens to generate
  • options.temperature — Controls randomness. Lower is more deterministic.
  • options.maxBatchSize — Maximum number of requests per batch. Larger inputs will be split into multiple batches.
  • options.pollingInterval — How often to check batch status
  • options.timeout — Maximum time to wait for batch completion (default 24 hours)
  • options.fallbackDeadline — If set, cancel incomplete batches after this time and run remaining requests synchronously. 0 = disabled.
  • options.metadata — Optional metadata to attach to the batch

Output shape

array of output items, each containing json with: response (assistant content string), fullResponse (raw API response), batchId (comma-separated batch IDs), customId (input index string), fallback (boolean). Errors include error message and code. Items with skipped empty prompts receive a default 200 response.

In batch mode, requests are grouped into JSONL batches, polled for completion, and incomplete requests beyond the fallback deadline run synchronously. In direct mode, each request is sent immediately with retry and exponential backoff.

Examples

Example 1: Batch mode with default settings

processingMode=batch (default), model=gpt-4o-mini, inputMode=simple, prompt provided via expression. Node chunks requests into batches of 100, uploads JSONL, polls for completion, and returns assistant messages per input item.

Example 2: Direct mode for low-latency needs

processingMode=direct, same model and input. Each request is sent immediately to /v1/chat/completions with retry on 429/5xx errors. Results returned immediately per item, no batch polling.

Links

Discussion