Why is CockroachDB's disk usage increasing even though I'm not writing to it?

Question

I started a CockroachDB process recently and noticed that its disk usage has continued increasing even though I stopped writing to it hours ago. What's going on? Is there any way to stop it from growing?

Answer 1

CockroachDB stores internal timeseries monitoring data into itself in order to power the graphs in its admin UI. This data accumulates for 30 days before it starts getting deleted. As a result, for the first 30 days or so of a cluster’s life you will see a steady increase in disk usage and the number of ranges in the cluster even if you aren’t writing data to it yourself.

As of the 1.0 release, there is no way to change the number of days before timeseries data gets truncated. As a workaround, however, you can start each node with the COCKROACH_METRICS_SAMPLE_INTERVAL environment variable set higher than its default of 10s to store fewer data points. For example, you could set it like COCKROACH_METRICS_SAMPLE_INTERVAL=1m to only collect data every 1 minute, which would result in storing 6x less timeseries data than the default setting. You could also set it to a very large value, such as COCKROACH_METRICS_SAMPLE_INTERVAL=87600h to only sample data once every 10 years (effectively never).