LLM Batchly icon

LLM Batchly

Create async LLM batches via the LLM Batchly public API

Actions4

Batch → Create

AI-generated

Summary

Create an asynchronous batch of LLM (Large Language Model) requests using the Batchly API with customizable input sources and parameters, optionally simplified output.

Inputs

  • Payload Source (required) — Select source for batch request payload: 'Node Parameters' to build from node fields; 'Incoming Item JSON' to use the entire JSON object of the incoming item as the batch request body; 'Image Generation' to create an image generation request with simplified fields.
  • Provider — Choose AI provider: Anthropic, Gemini, or OpenAI. Shown when Payload Source is 'parameters' or 'imageGeneration'.
  • Model Source — Select 'Catalog' to choose from the live Batchly model catalog or 'Custom' to manually enter model name. Shown when Payload Source is 'parameters' or 'imageGeneration'.
  • Model / Custom Model (required) — Select or enter the model name to use from the catalog or custom input. Shown depending on Model Source choice and Payload Source.
  • Callback URL (required) — HTTP URL where Batchly should POST completion data. Defaults to {{$execution.resumeUrl}} for webhook resume integration.
  • Execution Mode — Choose batch execution mode: 'Auto' for parallel realtime on small payloads and native batch for large; 'Native Batch' for provider's native batch API; 'Parallel Realtime' for small parallel requests with callback after all complete. For Image Generation, only 'Native Batch' is allowed.
  • Requests JSON (required) — JSON array of canonical message requests including 'custom_id' and 'messages'; used when Payload Source is 'parameters'.
  • Custom ID (required) — Stable identifier for the image generation request result; used when Payload Source is 'imageGeneration'.
  • Prompt (required) — Text prompt for image generation; used when Payload Source is 'imageGeneration'.
  • Image Size — Select size of generated image for image generation requests.
  • Quality — Image generation quality setting for OpenAI provider.
  • Output Format — Output image format (PNG, JPEG, WebP) for OpenAI image generation.
  • Output Compression — Optional compression percentage (0-100) for JPEG/WebP images; 0 to omit.
  • Background — Background setting (Auto, Opaque, Transparent) for generated images with OpenAI.
  • Force Image Generation — Force using OpenAI's image_generation tool instead of automatic model choice.
  • Store Provider Response — For OpenAI, whether the image generation response should be stored.
  • Additional Options — Advanced request options like response format override, structured output modes and schema, warmup, cost limits, time limit, cancel on caller abort, metadata JSON, and idempotency key; applicable when Payload Source is 'parameters'.
  • Simplify Output — If true, outputs a compact JSON including batch id, status, and request summary; otherwise, returns full detailed request and response.

Output shape

a single JSON object per input item

Output contains batch_id and status for each created batch; optionally includes detailed request and response data based on 'Simplify Output' setting. Errors per item are reported in the output JSON if the node is set to continue on failure.

Examples

Example 1: Create a batch from node parameters

User selects Payload Source 'Node Parameters', specifies Provider, Model, Requests JSON array, Callback URL, and optionally sets Execution Mode and Additional Options. The node submits the batch request and outputs the batch ID and status.

Example 2: Create a batch from raw incoming JSON

User sets Payload Source to 'Incoming Item JSON', passing a fully formed batch request body in the item JSON, which the node sends unmodified to Batchly.

Example 3: Create a batch for image generation

User selects Payload Source 'Image Generation', specifies Provider, Model (from catalog or custom), image prompt, custom ID, image size and optionally image quality and format. The node creates an image generation batch request submitted via the native batch mode.

Links

Discussion