Kafka Trigger Enhanced icon

Kafka Trigger Enhanced

Consume messages from a Kafka topic

Kafka Trigger Enhanced

AI-generated

Summary

Consumes messages from a Kafka topic and starts the workflow for each message. Configure a topic, consumer group, and required Kafka credentials; optional settings control decoding, returned fields, concurrency, message limits, and offset handling.

Inputs

  • Topic (required) — Kafka topic to consume from.
  • Group ID (required) — Kafka consumer group ID.
  • Kafka credentials (required) — Required kafkaawscred credentials provide broker addresses, client ID, SSL, and region. When the credential has no broker addresses, the node uses the KAFKA_BOOTSTRAP_SERVERS environment variable.
  • Use Schema Registry — When enabled, decode message values with Confluent Schema Registry; provide the required Schema Registry URL. Decode errors are logged and the current value is still emitted.
  • Options — Configure reading from the beginning, JSON parsing, optional headers, Kafka auto-commit thresholds and intervals, heartbeat and session timeouts, and maximum in-flight requests. Only Message emits the decoded value directly and is shown when JSON parsing is enabled. Parallel Processing allows concurrent workflow executions, limited by Max Concurrent Executions; parallel mode uses Kafka auto-commit.
  • Enhanced Options — Set Max Messages to pause the consumer after that many processed messages (0 means unlimited). Manual Offset Management commits each offset after successful workflow completion, with a configurable message timeout and retry count. After retries are exhausted, Auto Commit on Max Retries determines whether to commit and continue or disconnect the consumer. Manual offset management is unavailable with parallel processing.

Output shape

One workflow execution per consumed Kafka message; each execution receives one item.

By default, the item JSON contains message (the value as a string, or a parsed value when JSON parsing succeeds) and topic; headers is included when Return Headers is enabled and Kafka provides headers. With Only Message enabled, the item is the message value itself. If JSON parsing fails, the unparsed value is emitted. Schema Registry decoding runs after JSON parsing when both options are enabled.

Links

Discussion