2

Good day everyone,

I have set up a test environment with three virtual machines, to be able to deploy a ceph cluster. The machines are all identical with CentOS7 64bit. I install ceph on all of them, and ceph deploy on node one. I installed the initial monitor on ceph-node1, but when I try to install on ceph-node2, I get an error and I can't get past it. I am not sure how to troubleshoot this. Any ideas?

[root@ceph-node1 ~]# ceph-deploy mon create ceph-node2
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.25): /usr/bin/ceph-deploy mon create ceph-node2
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph-node2
[ceph_deploy.mon][DEBUG ] detecting platform for host ceph-node2 ...
[ceph-node2][DEBUG ] connected to host: ceph-node2
[ceph-node2][DEBUG ] detect platform information from remote host
[ceph-node2][DEBUG ] detect machine type
[ceph_deploy.mon][INFO  ] distro info: CentOS Linux 7.2.1511 Core
[ceph-node2][DEBUG ] determining if provided host has same hostname in remote
[ceph-node2][DEBUG ] get remote short hostname
[ceph-node2][DEBUG ] deploying mon to ceph-node2
[ceph-node2][DEBUG ] get remote short hostname
[ceph-node2][DEBUG ] remote hostname: ceph-node2
[ceph-node2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph-node2][DEBUG ] create the mon path if it does not exist
[ceph-node2][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-ceph-node2/done
[ceph-node2][DEBUG ] create a done file to avoid re-doing the mon deployment
[ceph-node2][DEBUG ] create the init path if it does not exist
[ceph-node2][DEBUG ] locating the `service` executable...
[ceph-node2][INFO  ] Running command: /usr/sbin/service ceph -c /etc/ceph/ceph.conf start mon.ceph-node2
[ceph-node2][DEBUG ] === mon.ceph-node2 ===
[ceph-node2][DEBUG ] Starting Ceph mon.ceph-node2 on ceph-node2...
[ceph-node2][WARNIN] Running as unit ceph-mon.ceph-node2.1460047679.725134876.service.
[ceph-node2][DEBUG ] Starting ceph-create-keys on ceph-node2...
[ceph-node2][INFO  ] Running command: systemctl enable ceph
[ceph-node2][WARNIN] ceph.service is not a native service, redirecting to /sbin/chkconfig.
[ceph-node2][WARNIN] Executing /sbin/chkconfig ceph on
[ceph-node2][INFO  ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph-node2.asok mon_status
[ceph-node2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph-node2][WARNIN] monitor: mon.ceph-node2, might not be running yet
[ceph-node2][INFO  ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph-node2.asok mon_status
[ceph-node2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph-node2][WARNIN] ceph-node2 is not defined in `mon initial members`
[ceph-node2][WARNIN] monitor ceph-node2 does not exist in monmap
[ceph-node2][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[ceph-node2][WARNIN] monitors may not be able to form quorum
Marian Boricean
  • 110
  • 1
  • 11
  • I have met the same problem. I overcome the problem by touch a file in that location. name it exactly as described in ERROR. – Tiina Jul 25 '16 at 06:14
  • Thanks Tiina. What location exactly? I can't figure it out from the error message. The error from "admin_socket" is confusing :D – Marian Boricean Jul 25 '16 at 09:41
  • It says "/var/run/ceph/ceph-mon.ceph-node2.asok" in your post. So this is the place I think. – Tiina Jul 25 '16 at 11:16
  • Ah.. that one! :) Thanks a lot. I'll give it a try. This was some time ago and I've given up "playing" with it. Now I'm excited to start again. – Marian Boricean Jul 25 '16 at 15:38
  • 1
    just like what i did. :D – Tiina Jul 25 '16 at 16:16
  • So Tiina, how is your experience so far with Ceph? What did you do exactly? :) – Marian Boricean Jul 30 '16 at 18:54
  • I used several VMs to build up this cluster. And use it like almost any IT system, seeing how to make it panic, and how to recover it. Digging its function and better understanding concepts like zones and regions. And see how it could be integrated or migrated with other systems. Openstack is one of such. So basically, playing with ceph brings me fun, that is also why I m doing this :) – Tiina Jul 31 '16 at 08:08
  • do you have an email address where I can write you? :) If you do maybe you can send it to marianboricean at gmail dot com – Marian Boricean Jul 31 '16 at 16:25
  • you may join ceph maillist. – Tiina Aug 01 '16 at 01:17
  • That's not what I was asking :). But OK. Thanks ;) – Marian Boricean Aug 01 '16 at 06:36

4 Answers4

2

The final solution should accord to the warn info: [ceph-node2][WARNIN] neither public_addr nor public_network keys are defined for monitors

So the solution is adding public_addr into ceph.conf file like this: public_network = 192.168.111.0/24

user7954943
  • 21
  • 2
  • I've tried by adding the mentioned line in ceph.conf file. But now , when I trigger ceph-deploy --overwrite-conf mon create-initial - I keep getting mon.cephserver not yet in quorum, server retrying. And it end with error "some monitors have still not reached quorum". – NightOwl19 Oct 25 '18 at 05:50
0

Ceph does not perform well on virtual servers. I've tested Luminous and Mimic on VMWare. You will not get relevant results (specially for I/O) as bare metal...

As for the error above you should have 2 network cards on each server. One public which you present to the customers and one private for the ceph to re-balance data.

Also remember to have FQDN names on your servers...

Bogdan Adrian Velica
  • 183
  • 9
0

I've faced with the same issue in Octopus which looks for me as mon container on node3 can't start with error:

no public_addr or public_network specified, and mon.node3 not present in monmap or ceph.conf

Unfortunately I've not figure out real technical reason why it happens and just delete daemon mon.node3

ceph orch daemon rm mon.node3 --force

and wait until ceph recreate it again.

Oleg Neumyvakin
  • 9,706
  • 3
  • 58
  • 62
0

I had same issue and all my setting was correct. after disable firewall and Selinux on all nodes, my issue Solved. BTW after change Selinux setting you should restart system to affect

Mohammad Ravanbakhsh
  • 2,244
  • 1
  • 15
  • 26