Kafka Trigger EnhancedInstall

Summary

Consume messages from a specified Kafka topic with enhanced control over offset management, message processing, and retry behavior.

Inputs

• Topic (required) — Kafka topic to consume from.
• Group ID (required) — Kafka consumer group ID to manage consumer group coordination.
• Use Schema Registry — Whether to deserialize messages using a Schema Registry.
• Schema Registry URL — URL of the Schema Registry used when Schema Registry is enabled.
• Options — Various basic Kafka consumer options such as automatic topic creation, auto commit thresholds and intervals, heartbeat interval, maximum requests, JSON parsing of message value, and parallel processing.
• Enhanced Options — Additional controls including reading messages from the beginning, session ID, maximum messages per batch, poll interval, manual offset management, message processing timeout and retry behavior including max retries and auto commit on timeout.

Output shape

a stream of messages emitted as individual JSON arrays containing message envelopes with fields such as 'message' (the message payload), 'topic' (topic name), and optionally 'headers' if requested.

Messages are emitted per each Kafka message consumed. When 'JSON Parse Message' is enabled, message values are parsed as JSON; if 'Use Schema Registry' is enabled, messages are deserialized via the configured Schema Registry. Offset commit is managed manually if enabled, meaning offsets commit after the Response node; retry logic and message timeout are managed according to the enhanced options. The node outputs continuously until stopped.