1

My Cassandra cluster (2 nodes) always timeouts when I select from table that have counters and filter by partition key.

cqlsh:test> select count(*) from costs_by_domain_and_format;

count
-------
2231

So, there's almost no data in it (2K rows). And when I query it:

cqlsh:test> select * from costs_by_domain_and_format where flight_id=6f0753b5-858d-44a0-a321-ce4f5869f58f;
Request did not complete within rpc_timeout.

It always time-outs. When I enabled tracing, it seems like it happened on Merging data from memtables and sstables:

Merging data from memtables and 4 sstables | 10:16:17,907 | xx.xx.xx.xx |           5600
Timed out; received 0 of 1 responses | 10:16:27,906 | xx.xx.xx.xx |       10004599

There is no problem to query this table without WHERE clause, like:

select * from costs_by_domain_and_format;

The table structure is shown below:

CREATE TABLE costs_by_domain_and_format (
  flight_id uuid,
  domain text,
  span_date timestamp,
  format text,
  cost counter,
  counter counter,
  PRIMARY KEY (flight_id, domain, span_date, format)
) WITH
 bloom_filter_fp_chance=0.010000 AND
 caching='KEYS_ONLY' AND
 comment='' AND
 dclocal_read_repair_chance=0.000000 AND
 gc_grace_seconds=864000 AND
 read_repair_chance=0.100000 AND
 replicate_on_write='true' AND
 populate_io_cache_on_flush='false' AND
 compaction={'class': 'SizeTieredCompactionStrategy'} AND
 compression={'sstable_compression': 'SnappyCompressor'};

Where is the source of this problem? Is that improper usage of counters? Or maybe incorrect table design? Or maybe this is some known issue in Cassandra (I use v1.2.10)??

Bartek
  • 183
  • 2
  • 10
  • 1
    Please check the Cassandra server logs to see if any node is logging errors or warnings as a result of that query. – Tupshin Harper Apr 07 '14 at 12:49
  • It appeared that the real problem was the same like in http://stackoverflow.com/questions/11623356/cassandra-server-throws-java-lang-assertionerror-decoratedkey-decorated and the solution was to call nodetool invalidatekeycache – Bartek Apr 07 '14 at 17:12

0 Answers0