1

I am trying to set up a Ceph cluster. I have 4 nodes - 1 admin-node, 1 monitor and 2 object storage devices. The installation guide I am using is given at the following location: http://ceph.com/docs/master/start/quick-ceph-deploy/.

When I am trying to add the initial monitor (step 5 in the guide), I am getting the following error:

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cloud-user/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.21): /usr/bin/ceph-deploy mon create-initial
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts worker-1-full
[ceph_deploy.mon][DEBUG ] detecting platform for host worker-1-full ...
[worker-1-full][DEBUG ] connection detected need for sudo
[worker-1-full][DEBUG ] connected to host: worker-1-full 
[worker-1-full][DEBUG ] detect platform information from remote host
[worker-1-full][DEBUG ] detect machine type
[ceph_deploy.mon][INFO  ] distro info: Ubuntu 14.04 trusty
[worker-1-full][DEBUG ] determining if provided host has same hostname in remote
[worker-1-full][DEBUG ] get remote short hostname
[worker-1-full][DEBUG ] deploying mon to worker-1-full
[worker-1-full][DEBUG ] get remote short hostname
[worker-1-full][DEBUG ] remote hostname: worker-1-full
[worker-1-full][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[worker-1-full][DEBUG ] create the mon path if it does not exist
[worker-1-full][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-worker-1-full/done
[worker-1-full][DEBUG ] create a done file to avoid re-doing the mon deployment
[worker-1-full][DEBUG ] create the init path if it does not exist
[worker-1-full][DEBUG ] locating the `service` executable...
[worker-1-full][INFO  ] Running command: sudo initctl emit ceph-mon cluster=ceph id=worker-1-full
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[worker-1-full][WARNIN] monitor: mon.worker-1-full, might not be running yet
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[worker-1-full][WARNIN] monitor worker-1-full does not exist in monmap
[worker-1-full][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[worker-1-full][WARNIN] monitors may not be able to form quorum
[ceph_deploy.mon][INFO  ] processing monitor mon.worker-1-full
[worker-1-full][DEBUG ] connection detected need for sudo
[worker-1-full][DEBUG ] connected to host: worker-1-full 
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.worker-1-full monitor is not yet in quorum, tries left: 5
[ceph_deploy.mon][WARNIN] waiting 5 seconds before retrying
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.worker-1-full monitor is not yet in quorum, tries left: 4
[ceph_deploy.mon][WARNIN] waiting 10 seconds before retrying
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.worker-1-full monitor is not yet in quorum, tries left: 3
[ceph_deploy.mon][WARNIN] waiting 10 seconds before retrying
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.worker-1-full monitor is not yet in quorum, tries left: 2
[ceph_deploy.mon][WARNIN] waiting 15 seconds before retrying
[worker-1-full][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.worker-1-full.asok mon_status
[worker-1-full][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.worker-1-full monitor is not yet in quorum, tries left: 1
[ceph_deploy.mon][WARNIN] waiting 20 seconds before retrying
[ceph_deploy.mon][ERROR ] Some monitors have still not reached quorum:
[ceph_deploy.mon][ERROR ] worker-1-full

"worker-1-full" is the node I am trying to set up as my monitor. The command I used is:

"ceph-deploy mon create-initial". Please help. Thanks in advance!

  • What about the missing file: /var/run/ceph/ceph-mon.worker-1-full.asok? – Deer Hunter Jan 14 '15 at 5:06
  • That's what - I don't know how to generate it. I am doing it exactly as it is given in the link. – user3033194 Jan 14 '15 at 10:31
  • I had this problem and I got that my / (roo) is full – Mansour Apr 12 '16 at 12:33
  • I think that it's about directory permission. Would you check out your directory (/var/lib/ceph/mon and /var/run/ceph) permision as ceph:ceph user ? especially, check out file&dir. in /var/lib/ceph/mon directory. If you execute ceph-deploy command as root user, may be created /var/lib/ceph/mon directory as root user. I got it that "chown -R ceph:ceph /var/lib/ceph/mon". – neotusca Jun 10 '16 at 6:18
6

I faced the same errors was able to resolve the issue by adding my other ceph node's hostname & IpAdrress and by adding "public_network ="

The sections which I tweaked in ceph.conf are:

mon_initial_members =
mon_host =
public_network =

cat /etc/ceph/ceph.conf

[global]
fsid = 33cb5c76-a685-469e-8cdd-fee7c98c3f4d
mon_initial_members = ceph1,ceph2
mon_host = 192.168.61.39,192.168.61.40
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
public_network = 192.168.61.0/24

And the running the command:

$ ceph-deploy --overwrite-conf mon create <ceph-node>
2

It seems that names of your nodes differ from their real hostnames, so that file /var/run/ceph/ceph-mon.worker-1-full.asok has the incorrect name. I had changed name of hosts and that worked for me.

0

Does the /var/run/ceph directory exist? If not, then Ceph probably won't be able to create the .asok (admin socket) file.

  • the directory do not exist. how to do with that? – aircraft Jul 19 '17 at 15:26
0

You could check the MON log at worker-1-full:/var/log/ceph/ceph-mon.worker-1-full.log. If the MON doesn't come up or cannot create the socket, something should be logged over there.

0

In my case it was caused by the previous command failing, which was not detected by the script:

# ceph-deploy mon create-initial
...
[ceph-server-1][DEBUG ] create the init path if it does not exist
[ceph-server-1][INFO  ] Running command: systemctl enable ceph.target
[ceph-server-1][INFO  ] Running command: systemctl enable ceph-mon@ceph-server-1
[ceph-server-1][INFO  ] Running command: systemctl start ceph-mon@ceph-server-1
[ceph-server-1][INFO  ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph-server-1.asok mon_status
[ceph-server-1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory

With journalctl -xe I found:

Jul 05 12:12:29 ceph-server-1.novalocal ceph-mon[27403]: 2019-07-05 12:12:29.389 7f7b5aed7180 -1 rocksdb: IO error: while open a file for lock: /var/lib/ceph/mon/ceph-ceph-server-1/store.db/LOCK: Permission denied

To solve:

# chown -R ceph:ceph /var/lib/ceph/mon/ceph-ceph-server-1/store.db/
# chown    ceph:ceph /var/lib/ceph/mon/ceph-ceph-server-1/keyring

Now I have a happy Ceph.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.