Recently I restart one of the cassandra node (which is a seed node) and got following errors, the weird thing is that previously I start the cluster with token set to be 256, now I'm getting a "cannot change token from 512 to 256" error.
DEBUG [localhost-startStop-1] 2019-10-03 21:25:57,574
YamlConfigurationLoader.java:102 - Loading settings from
file:/mnt/ccm/cassandra-dev-snapshot-test/node1/conf/cassandra.yaml
DEBUG [localhost-startStop-1] 2019-10-03 21:25:57,592
StorageService.java:899 - Bootstrap variables: false false true true
ERROR [localhost-startStop-1] 2019-10-03 21:25:57,598
CassandraDaemon.java:727 - Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Cannot change
the number of tokens from 512 to 256 at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1006)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:753)
Anyone experience this before? The only thing I found online is when people try to change from vnodes to single token. And the solution is to completely delete the data and restore it from snapshot.
EDIT 1:
Cassandra version: 3.0.15
cassandra.yaml:
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
auto_bootstrap: false
auto_snapshot: true
batch_size_fail_threshold_in_kb: 50
batch_size_warn_threshold_in_kb: 5
batchlog_replay_throttle_in_kb: 1024
cas_contention_timeout_in_ms: 1000
client_encryption_options:
enabled: false
keystore: conf/.keystore
keystore_password: cassandra
optional: false
cluster_name: cassandra-snapshot-test
column_index_size_in_kb: 64
commit_failure_policy: stop
commitlog_directory: /mnt/ccm/cassandra-snapshot-test/node1/commitlogs
commitlog_segment_size_in_mb: 32
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
compaction_large_partition_warning_threshold_mb: 100
compaction_throughput_mb_per_sec: 16
concurrent_counter_writes: 32
concurrent_materialized_view_writes: 32
concurrent_reads: 32
concurrent_writes: 32
counter_cache_save_period: 7200
counter_cache_size_in_mb: null
counter_write_request_timeout_in_ms: 5000
cross_node_timeout: false
data_file_directories:
- /mnt/ccm/cassandra-snapshot-test/node1/data0
disk_failure_policy: stop
dynamic_snitch_badness_threshold: 0.1
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_update_interval_in_ms: 100
enable_scripted_user_defined_functions: false
enable_user_defined_functions: false
endpoint_snitch: SimpleSnitch
gc_warn_threshold_in_ms: 1000
hinted_handoff_enabled: true
hinted_handoff_throttle_in_kb: 1024
hints_directory: /mnt/ccm/cassandra-snapshot-test/node1/hints
hints_flush_period_in_ms: 10000
incremental_backups: false
index_summary_capacity_in_mb: null
index_summary_resize_interval_in_minutes: 60
initial_token: null
inter_dc_tcp_nodelay: false
internode_compression: all
key_cache_save_period: 14400
key_cache_size_in_mb: null
listen_address: 172.27.17.176
max_hint_window_in_ms: 10800000
max_hints_delivery_threads: 2
max_hints_file_size_in_mb: 128
memtable_allocation_type: heap_buffers
native_transport_port: 9042
num_tokens: 256
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
permissions_validity_in_ms: 2000
range_request_timeout_in_ms: 10000
read_request_timeout_in_ms: 5000
request_scheduler: org.apache.cassandra.scheduler.NoScheduler
request_timeout_in_ms: 10000
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
row_cache_save_period: 0
row_cache_size_in_mb: 0
rpc_address: *IP*
rpc_keepalive: true
rpc_port: 9160
rpc_server_type: sync
saved_caches_directory: /mnt/ccm/cassandra-snapshot-test/node1/saved_caches
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: *IP*
server_encryption_options:
enabled: false
internode_encryption: none
keystore: conf/.keystore
keystore_password: cassandra
optional: false
truststore: conf/.truststore
truststore_password: cassandra
snapshot_before_compaction: false
ssl_storage_port: 7001
sstable_preemptive_open_interval_in_mb: 50
start_native_transport: true
start_rpc: false
storage_port: 7000
thrift_framed_transport_size_in_mb: 15
tombstone_failure_threshold: 100000
tombstone_warn_threshold: 1000
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
trickle_fsync: false
trickle_fsync_interval_in_kb: 10240
truncate_request_timeout_in_ms: 60000
unlogged_batch_across_partitions_warn_threshold: 10
windows_timer_interval: 1
write_request_timeout_in_ms: 2000
