58

How am I supposed to bot a new Cassandra node when I get this error?

INFO [SSTableBatchOpen:1] 2014-02-25 01:51:17,132 SSTableReader.java (line 223) Opening /var/lib/cassandra/data/system/local/system-local-jb-5 (5725 bytes)
ERROR [main] 2014-02-25 01:51:17,377 CassandraDaemon.java (line 237) Fatal exception during initialization
org.apache.cassandra.exceptions.ConfigurationException: Saved cluster name Test Cluster != configured name thisisstupid
        at org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:542)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:233)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:462)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:552)

Name of cluster in the cassandra.yaml file is:

cluster_name: 'thisisstupid'

How do I resolve?

Tampa
  • 75,446
  • 119
  • 278
  • 425

7 Answers7

64

You can rename the cluster without deleting data by updating it's name in the system.local table (but you have to do this for each node...)

cqlsh> UPDATE system.local SET cluster_name = 'test' where key='local';
# flush the sstables to persist the update.
bash $ ./nodetool flush

Finally you need to rename the cluster to the new name in cassandra.yaml (again on each node)

Greg Dubicki
  • 5,983
  • 3
  • 55
  • 68
Lyuben Todorov
  • 13,987
  • 5
  • 50
  • 69
  • 1
    what dir's are those? – Tampa Feb 25 '14 at 07:46
  • Check for the `data_file_directories`, `commitlog_directory` (and in 2.1 flush_directory) settings in `cassandra.yaml` – Lyuben Todorov Feb 25 '14 at 07:48
  • 1
    dirs are in /var/lib/cassandra/commitlog /var/lib/cassandra/data so....delete or flush contents of the dirs? – Tampa Feb 25 '14 at 07:51
  • 2
    @Tampa del them, but this will delete all your data aswell. You are basically completely wiping the cluster. – Lyuben Todorov Feb 25 '14 at 07:51
  • nice @Tampa, deleting those is the right move in the case you're just setting up a new node and forgot to change the name in the config :) – stantonk Mar 02 '15 at 07:28
  • 3
    I believe `bash $ ./nodetool flush` should be `bash $ ./nodetool flush system`. For details, see [here](https://support.datastax.com/hc/en-us/articles/205289825-Change-Cluster-Name-) – hammadian Oct 12 '16 at 19:25
  • 9
    If Cassandra couldn't be started due to the stated error, then we obviously can't use the cqlsh approach. We would then need to delete the files at system keyspace and restart the service. – user2275460 Jan 13 '17 at 17:11
  • @user2275460 Great, but how do you do that without data loss? Also the exception only appears in the new node being added, the others can start up, thus that is where the cql update needs to occur. You propagate the change to all your nodes, then start the new node and the exception will be resolved. – Lyuben Todorov Feb 12 '17 at 17:57
  • @user2275460 I was unable to start the cluster due to this issue. I've deleted the files at /var/lib/cassandra/data/system. Still see the issue. Is there any work around? – spark_dream Sep 19 '17 at 20:32
31

The above commands with UPDATE SET cluster_name does not work for me. I found the working is to follow the instructions from the DataStax documentation on Initializing a multiple node cluster:

sudo service cassandra stop
sudo rm -rf /var/lib/cassandra/data/system/*
sudo vi /etc/cassandra/cassandra.yaml, setup the proper parameters
sudo service cassandra start
nodetool status

For some good cluster node setups, I found this blog post to be very useful.

muru
  • 4,723
  • 1
  • 34
  • 78
Ryan X
  • 575
  • 7
  • 5
17

I had same issue with Datastax4.7 enterprise edition. I resolved it with above instructions:

Change the cluster name back to "test Cluster"

start the cluster:

cqlsh> UPDATE system.local SET cluster_name = 'Cassendra Cluster' where key='local';
cqlsh> exit;
$ nodetool flush system

Stop the cluster:

$sudo service dse stop;sudo service datastax-agent stop

Edit the file:

$ sudo vi /etc/dse/cassandra/cassandra.yaml
cluster_name: 'Cassendra Cluster'

Start the cluster:

$sudo service dse start;sudo service datastax-agent start

Check Installation log:

 $ cat /var/log/cassandra/output.log
Leena Samuel
  • 67
  • 2
  • 12
K Murthy
  • 171
  • 1
  • 3
9

For Cassandra 3.0.5, I tired the answer by Lyuben Todorov but it was missing a key part:

bash $ ./nodetool flush

should be:

bash $ ./nodetool flush system

as in here

Community
  • 1
  • 1
hammadian
  • 323
  • 2
  • 9
3

For Cassandra 3.7 you should replace the ./nodetool flush with ./nodetool flush system. Otherwise it won't work.

MikeT
  • 51,415
  • 16
  • 49
  • 68
hxg
  • 31
  • 2
1

If cassandra is not running, you can wipe the data folder.

rm -rf /usr/lib/cassandra/data/*

This will clear the system tables which had the wrong cluster name. \ When your restart cassandra with the correct cluster name in your cassandra.yaml everything will be regenerated and you should be good.

alexander.polomodov
  • 5,396
  • 14
  • 39
  • 46
Noah Ellman
  • 179
  • 1
  • 4
  • 3
    Deleting all of your data does fix the problem, but I am not sure this is the best thing to advise people. – Jay Sep 21 '17 at 00:48
  • 1
    I think it's unfair to downvote this answer. Early in the deployment process, flushing the data might be much easier than having to bring back up the cluster on it's previous name just to change it via cqlsh. – sph21 Aug 29 '18 at 00:55
0

For Bitnami certified Cassandra - 3.11.5

Here's the solution that worked for me:

  1. Stop the Cassandra service
sudo /opt/bitnami/ctlscript.sh stop cassandra
  1. Remove the system data
sudo rm -rf /opt/bitnami/cassandra/data/data/system/*
  1. Start the Cassandra Service
sudo /opt/bitnami/ctlscript.sh start cassandra
Tarang Bhalodia
  • 1,185
  • 11
  • 21