1

Been using a 6GB dataset with each source record being ~1KB in length when I accidentally added an index on a column that I am pretty sure has a 100% cardinality.

Tried dropping the index from cqlsh but by that point the two node cluster had gone into a run away death spiral with loadavg surpassing 20 on each node and cqlsh hung on the drop command for 30 minutes. Since this was just a test setup, I shut-down and destroyed the cluster and restarted.

This is a fairly disconcerting problem as it makes me fear a scenario where a junior developer is on a production cluster and they set an index on a similar high cardinality column. I scanned through the documentation and looked at the options in nodetool but there didn't seem to be anything along the lines of "abort job or abort building index".

Test environment: 2x m1.xlarge EC2 instances with 2 Raid 0 ephemeral disks Dataset was 6GB, 1KB per record.

My question in summary: Is it possible to abort the process of building a secondary index AND or possible to stop/postpone running builds (indexing, compaction) for a later date.

David
  • 17,673
  • 10
  • 68
  • 97

1 Answers1

2

nodetool -h node_address stop index_build

See: http://www.datastax.com/docs/1.2/references/nodetool#nodetool-stop

Dirk Lachowski
  • 3,121
  • 4
  • 40
  • 66
  • For some reason when I tried that during my test, it didn't actually stop the secondary index build. Not sure if it was just how screwed up my test cluster had gotten before I tried to intercede or if there was something else broken. – David Jul 12 '13 at 19:46