0

I have 4 node ceph architecture as shown in the figure. Ceph Architecture Ceph.conf contains

[global]
fsid = 23923667-d7af-4138-a6e5-2e38fb999e2d
max open files = 131072
mon_initial_members = host1, host2
mon host = 10.xx.xx.1,10.xx.xx.2
public_network = 10.xx.xx.xx/27
cluster_network = 10.xx.xx.xx/27

The first monitor has been attached using the following command

ceph-deploy mon create-initial

Second monitor has been added to the cluster by the following command

ceph-deploy mon add 10.xx.xx.2

We are testing high availability cases with this setup.

Case 1: I brought down the node 10.xx.xx.2 and tested my connection. Ceph was up and responding with the monitor 10.xx.xx.1

Case 2: I brought down the node 10.xx.xx.1. But I am unable to access the ceph cluster via 10.xx.xx.2.

Is there any other way to create the setup so that the second node becomes master when the first node is down ?

[edit1]

The scenario is

 intial monitor node = mon.a
 additional monitor node =mon.b,mon.c

When I bring down mon.b and mon.c ceph is working fine.

But when I bring down mon.a entire ceph cluster gets disturbed and ceph is not responding.

satheesh
  • 11
  • 4
  • what is the behavior if you ceph status in the failed mon? – Tiina Aug 30 '16 at 00:54
  • I could not check ceph status if the monitor (10.xx.xx.x2) is down. I am testing it by shutting down one server. – satheesh Aug 30 '16 at 08:10
  • We need more information on how you're actually testing this. Let's call 10.x.x.x.1 mon.a and 10.x.x.x.2 mon.b. With both monitors up, post the results of `ceph -s`, with mon.a down post the results of `ceph -s` and then turn mon.a back on and bring down mon.b and test that. Ideally you'd want a mon.c at a third site or separate piece of kit acting as an arbitrator. Technically you should be using a minimum of 3, as a cluster will only function when it's considered quorate, i.e. has at least more than half of the initial mon nodes. That will give you degraded access until the issue is resolved – b0bu Sep 06 '16 at 13:43
  • I have edited my post. Please refer and let me know. – satheesh Sep 08 '16 at 06:37
  • How many mon do you have? 2? you should post the output of # ceph mon dump – alsotoes Feb 21 '17 at 08:04

1 Answers1

1

If you have two MON, either MON disappear will not have Quorum. Therefore you will not have a running state if one MON disappear. So yes, you always need odd number for MON total. 1,3,5, etc. I'm not sure why you have Quorum without any one for MON. But if you really do, ceph -s show you how does it get Quorum.

AvengerMoJo
  • 118
  • 6