I am setting up a riak cluster of five physical nodes. The other four are fine with all tests except one fails the admin-riak test. The cluster state on several riak-admin commands is shown below
do-admin@DBNode1:~$ sudo riak-admin member-status
=============== Membership ============================
Status Ring Pending Node
-------------------------------------------------------------------------------
valid 20.3% -- 'riak@dbnode1.do.ug'
valid 20.3% -- 'riak@dbnode2.do.ug'
valid 20.3% -- 'riak@dbnode3.do.ug'
valid 20.3% -- 'riak@dbnode4.do.ug'
valid 18.8% -- 'riak@dbnode5.do.ug'
-------------------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
dot-admin@DBNode1:~$ sudo riak-admin ring-status
================================== Claimant ===================================
Claimant: 'riak@dbnode2.do.ug'
Status: up
Ring Ready: true
============================== Ownership Handoff ==============================
No pending changes.
============================== Unreachable Nodes ==============================
All nodes are up and reachable
do-admin@DBNode1:~$ sudo riak-admin cluster status
---- Cluster Status ----
Ring ready: true
+------------------------+------+-------+-----+-------+
| node |status| avail |ring |pending|
+------------------------+------+-------+-----+-------+
| riak@dbnode1.do.ug |valid | up | 20.3| -- |
| (C) riak@dbnode2.do.ug |valid | up | 20.3| -- |
| riak@dbnode3.do.ug |valid | up | 20.3| -- |
| riak@dbnode4.do.ug |valid | up | 20.3| -- |
| riak@dbnode5.do.ug |valid | up | 18.8| -- |
+------------------------+------+-------+-----+-------+
Key: (C) = Claimant; availability marked with '!' is unexpected
do-admin@DBNode1:~$ curl -v http://dbnode1.dot.ug:8098/types/default/props
* Hostname was NOT found in DNS cache
* Trying 192.168.172.38...
* Connected to dbnode1.dot.ug (192.168.172.38) port 8098 (#0)
> GET /types/default/props HTTP/1.1
> User-Agent: curl/7.35.0
> Host: dbnode1.dotshule.ug:8098
> Accept: */*
>
< HTTP/1.1 200 OK
< Vary: Accept-Encoding
* Server MochiWeb/1.1 WebMachine/1.10.5 (jokes are better explained) is not blacklisted
< Server: MochiWeb/1.1 WebMachine/1.10.5 (jokes are better explained)
< Date: Sat, 17 Jan 2015 21:05:22 GMT
< Content-Type: application/json
< Content-Length: 428
<
* Connection #0 to host dbnode1.dotshule.ug left intact
{"props":{"allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dvv_enabled":false,"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":"quorum","rw":"quorum","small_vclock":50,"w":"quorum","young_vclock":20}}
dot-admin@DBNode1:~$ sudo riak-admin test
Node 'riak@dbnode1.dot.ug ' is not reachable from 'riak_test@dbnode1.dot.ug'.
After all those tests with the same results on all nodes except for the **riak-admin test ** which is like this on all other nodes. For node three
dot-admin@DBNode3:~$ sudo riak-admin test
Successfully completed 1 read/write cycle to 'riak@dbnode3.dotshule.ug'
My doubt is whether this cluster is ready to used to store data because on this Basho website resource, they say you can use any of the methods to test whether the node is ready. They do not say that the node is still fine if one method success and the other fails. So I am stuck on whether to go ahead to use the cluster or not. Surprisingly this node has succeeded on all operations to join it in the cluster!! I have tried creating this node from scratch again but that has not helped!
For any help, I will be glad.