Description
Problem
There's no simple way to purge a topic that's by using kafkactl or kafka-admin. The goal is to be able to easily purge a topic from all the records it contains !
Suggestion
The user should be able to do :
kafkactl purge topic <topicName> <--force, --retention-ms=durationMS>Alternatives Considered
There's two way that could be used to purge a topic from its content one being easier to execute that the other
-
Store in memory the topic.yaml, delete the topic, re-apply the topic.yaml in memory (this avoids a lot of overhead with the other method)
-
Set the topic retention.ms temporarily to a low value like 10ms to delete all records that are older than the retention.ms value, but this implies that the user needs to reset the offsets of all the consumer groups subscribed to the topics that will purged
While the second way requires a bit more of setting up, it could offer more than purging a whole topic we could imagine that this could be used to purge only old message from a topic while keeping the more recent records unscathed !
What are your thoughts on this ? I'll contribute if the issue is deems useful :)