TIL: Kafka CLI and group offset

Today, I discovered that when utilizing kafka-console-consumer and including the --group flag without specifying --from-beginning, the offset is set to the latest message. Subsequently, adding --from-beginning to later commands will be disregarded. Meaning it’ll only consume new events.

This information is beneficial during debugging and maintenance operations. The offset must be reset to rectify this, which can be achieved through specific steps.

$ kafka-consumer-groups \
  --bootstrap-server=[Bootstrap-Server]:9092 \
  --topic [Topic] \
  --reset-offsets --to-earliest --execute \
  --group [Consumer-Group] \
  --command-config [Authentication.conf]