1

I set up nats streaming server in cluster mode (according to instructions found here) and I noticed raft.log is growing really fast when I benchmark it with a load around 200msg/s where message size is ~1MB.
I use default channel settings:

---------- Store Limits ----------
Channels:                  100 *
--------- Channels Limits --------
Subscriptions:          1000 *
Messages     :       1000000 *
Bytes        :     976.56 MB *
Age          :     unlimited *
Inactivity   :     unlimited *

Are there any configuration parameters that could help limiting size or truncating the file?

Community
  • 1
  • 1
Jakub Bibro
  • 1,570
  • 13
  • 17

1 Answers1

2

We use go.etcd.io/boltdb for the RAFT log backend and this does not offer truncation/compaction. The log contains all data replicated in the NATS Streaming system, but when nodes are in sync, snapshots should occur at some random (but no less than 2 minutes) interval. This should allow RAFT to delete entries from its log, which mean that when adding new data, the free space should be reused and not result in growth of the main log file. Of course, if the rate/amount of data added surpass the free space from the previous snapshot, the raft.log file will grow.

I. Kozlovic
  • 796
  • 3
  • 3