0

I am trying to setup some persistent data kafka topics where none of the messages expire so I can do backfills and run ksqldb. I understand that retention.ms set to -1 is the most important parameter but could anyone suggest any other settings that would affect performance specifically related to persisting data indefinitely. Currently using to setup the topics with strimzi:

    retention.ms: -1
    retention.bytes: -1
    segment.bytes: 1073741824
    cleanup.policy: compact
    min.cleanable.dirty.ratio: 0.5
    preallocate: true
robcxyz
  • 68
  • 1
  • 6
  • 1
    This might be a little broad, and dependent on your use case + data.. – pyeR_biz Jul 30 '21 at 10:54
  • The dirty ratio is the only other semi-important one for compaction, and depends on how repetitive your keys are or how frequent you want compaction to happen – OneCricketeer Jul 30 '21 at 18:15

0 Answers0