1

We have been historically using STCS as the compaction strategy for our Cassandra table, which was resulting in too many tombstones. Given data in the table is append only and uses a TTL of 30 days (No manual updates or deletes), we switched to TWCS with compaction_window_size of 1 day. Which resulted in high cpu usage and LiveSSTableCount, along with high read latencies from the cluster.

Dashboard snapshots below.
Strategy change aligns with the inflection in the graphs.

High CPU Usage

Increased FD

Increased Block IO

Is this expected when switching from STCS to TWCS? Can we tune the parameters here to make sure this doesn't affect the reads from the cluster?

More details of the table schema.

CREATE TABLE Foo (
user_uuid uuid,
ts bigint,
... //skipped a few columns
PRIMARY KEY (user_uuid, ts, event_uuid)
) WITH CLUSTERING ORDER BY (ts DESC, event_uuid ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 'compaction_window_size': '24', 'compaction_window_unit': 'HOURS', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 2592000
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
Abhinav
  • 1,346
  • 12
  • 25
  • Whats your query? The optimization to exclude sstables outside the min/max clustering key (ts) is important with TWCS especially if your partition is unbounded like that. – Chris Lohfink Jan 30 '20 at 17:24
  • We are keeping 30 days of data and reading all 30 days of data in queries. Can you please elaborate on "exclude sstables outside the min/max clustering key". Given I need all of data for a particular user, I can't put any filters on my reads. Is there anything else I am missing? – Abhinav Jan 30 '20 at 17:43

0 Answers0