2

I am running a Ceph Cluster (pacific) consisting of 3 nodes. I have 3 Monitor Daemons started (one on each node).

Just for some test I wanted to remove node3 out of the list of monitors with the command

sudo ceph mon remove node3

The mon disappeared in the Dashboard and also when I call ceph status command:

$ sudo ceph status
  cluster:
    id:     xxxxx-yyyyy-zzzzzzz
    health: HEALTH_WARN
            1 failed cephadm daemon(s)
 
  services:
    mon: 2 daemons, quorum node1,node2 (age 53m)
    mgr: node1.gafnwm(active, since 102m), standbys: node2.jppwxo
    osd: 3 osds: 3 up (since 15m), 3 in (since 96m)

But now I found no way to restart or add the monitor again into my cluster. I tried:

$ sudo ceph orch daemon add mon node3:10.0.0.4
Error EINVAL: name mon.node3 already in use

What does this mean? Why is the name mon.node3 already in use even if I can't see it?

Ralph
  • 4,500
  • 9
  • 48
  • 87
  • 3
    If you run `cephadm ls` on that node you will see the previous daemon. Remove it with `cephadm rm-daemon --name mon.`. If that worked you'll most likely be able to redeploy the mon again. – eblock Oct 08 '21 at 06:39
  • 1
    The mon was listed in the 'cephadm ls' resultlist. I removed it with `sudo cephadm rm-daemon --name mon.node3 --fsid xxxx-yyyy-zzz --force` and the daemon disapeard in the `cephadm ls`. In the web dashboard it is still listed (strange) and still I am not able to add it. I got the same error message `Error EINVAL: name mon.node3 already in use` – Ralph Oct 08 '21 at 07:01
  • 1
    I tried a reboot of the node and than the mon disappeared also in the web dashboard and now I was able to add the mon again ! Thanks a lot for your help. – Ralph Oct 08 '21 at 07:06
  • 1
    Sometimes cephadm needs some time to update its information, most operations are delayed, so it could also have worked just with some waiting instead of reboot, but great that it worked that way, too. – eblock Oct 08 '21 at 07:25
  • 1
    I suppose you could speed up the update if you restart mgr daemon. Or I've just been lucky. – ogurets May 24 '23 at 23:04

0 Answers0