Kafka Trigger Enhanced
AI-generatedSummary
Consumes messages from a Kafka topic and starts the workflow for each message. Configure a topic, consumer group, and required Kafka credentials; optional settings control decoding, returned fields, concurrency, message limits, and offset handling.
Inputs
- Topic (required) — Kafka topic to consume from.
- Group ID (required) — Kafka consumer group ID.
- Kafka credentials (required) — Required
kafkaawscredcredentials provide broker addresses, client ID, SSL, and region. When the credential has no broker addresses, the node uses theKAFKA_BOOTSTRAP_SERVERSenvironment variable. - Use Schema Registry — When enabled, decode message values with Confluent Schema Registry; provide the required Schema Registry URL. Decode errors are logged and the current value is still emitted.
- Options — Configure reading from the beginning, JSON parsing, optional headers, Kafka auto-commit thresholds and intervals, heartbeat and session timeouts, and maximum in-flight requests.
Only Messageemits the decoded value directly and is shown when JSON parsing is enabled.Parallel Processingallows concurrent workflow executions, limited byMax Concurrent Executions; parallel mode uses Kafka auto-commit. - Enhanced Options — Set
Max Messagesto pause the consumer after that many processed messages (0 means unlimited).Manual Offset Managementcommits each offset after successful workflow completion, with a configurable message timeout and retry count. After retries are exhausted,Auto Commit on Max Retriesdetermines whether to commit and continue or disconnect the consumer. Manual offset management is unavailable with parallel processing.
Output shape
One workflow execution per consumed Kafka message; each execution receives one item.
By default, the item JSON contains
message(the value as a string, or a parsed value when JSON parsing succeeds) andtopic;headersis included when Return Headers is enabled and Kafka provides headers. With Only Message enabled, the item is the message value itself. If JSON parsing fails, the unparsed value is emitted. Schema Registry decoding runs after JSON parsing when both options are enabled.