Description of problem: As subject Version-Release number of selected component (if applicable): systemd-239-6.git9f3aed1.fc29.x86_64 ceph-base-12.2.9-1.fc29.x86_64 ceph-deploy-1.5.32-8.fc29.noarch How reproducible: 100% Steps to Reproduce: 1. Prepare a ceph cluseter as following steps: i. # ceph-deploy new --public-network=NETWORK --cluster-network=NETWORK MON ii. # ceph-deploy install admin MON OSD1 OSD2 iii. # ceph-deploy mon create-initial iv. Prepare /dev/vdb1 as OSDs in host OSD1 OSD2 v. # ceph-deploy osd create OSD1:/dev/vdb1 OSD2:/dev/vdb1 vi. # ceph-deploy osd activate OSD1:/dev/vdb1 OSD2:/dev/vdb1 [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] Traceback (most recent call last): [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/sbin/ceph-disk", line 11, in <module> [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')() [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5736, in run [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] main(sys.argv[1:]) [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5687, in main [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] args.func(args) [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3829, in main_activate [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] osd_id=osd_id, [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3376, in start_daemon [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] raise Error('ceph osd start failed', e) [vm-74-208.lab.eng.pek2.redhat.com][WARNIN] ceph_disk.main.Error: Error: ceph osd start failed: Command '['/usr/bin/systemctl', 'disable', 'ceph-osd@0', '--runtime']' retur ned non-zero exit status 1 [vm-74-208.lab.eng.pek2.redhat.com][ERROR ] RuntimeError: command returned non-zero exit status: 1 [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /dev/vdb1 2. Manually run ceph-disk command on OSD1 host: # # /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /dev/vdb1 main_activate: path = /dev/vdb1 get_dm_uuid: get_dm_uuid /dev/vdb1 uuid path is /sys/dev/block/252:17/dm/uuid command: Running command: /usr/sbin/blkid -o udev -p /dev/vdb1 command: Running command: /sbin/blkid -p -s TYPE -o value -- /dev/vdb1 command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs mount: Mounting /dev/vdb1 on /var/lib/ceph/tmp/mnt.HUgGTT with options noatime,inode64 command_check_call: Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/vdb1 /var/lib/ceph/tmp/mnt.HUgGTT command: Running command: /usr/sbin/restorecon /var/lib/ceph/tmp/mnt.HUgGTT activate: Cluster uuid is bdc9c6d8-e2f7-43af-817e-0e0b43b626fc command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid activate: Cluster name is ceph activate: OSD uuid is cc96300c-e269-430b-bc9c-6caab79535c2 activate: OSD id is 0 activate: Marking with init system systemd command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/tmp/mnt.HUgGTT/systemd command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.HUgGTT/systemd activate: ceph osd.0 data dir is ready at /var/lib/ceph/tmp/mnt.HUgGTT mount_activate: ceph osd.0 already mounted in position; unmounting ours. unmount: Unmounting /var/lib/ceph/tmp/mnt.HUgGTT command_check_call: Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.HUgGTT start_daemon: Starting ceph osd.0... command_check_call: Running command: /usr/bin/systemctl disable ceph-osd@0 command_check_call: Running command: /usr/bin/systemctl disable ceph-osd@0 --runtime --runtime cannot be used with disable Traceback (most recent call last): File "/usr/sbin/ceph-disk", line 11, in <module> load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')() File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5736, in run main(sys.argv[1:]) File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5687, in main args.func(args) File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3829, in main_activate osd_id=osd_id, File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3376, in start_daemon raise Error('ceph osd start failed', e) ceph_disk.main.Error: Error: ceph osd start failed: Command '['/usr/bin/systemctl', 'disable', 'ceph-osd@0', '--runtime']' returned non-zero exit status 1 Actual results: As above Expected results: ceph-disk cmd runs well. Additional info: This error happened because systemd doesn't support disable verb with --runtime option since systemd-239. Here is the code commit of this change: commit 4910b35078ad24dcbc63f372b2fee087640201d0 Author: Zbigniew Jędrzejewski-Szmek <zbyszek.pl> Date: Thu May 31 15:28:07 2018 +0200 systemctl: when removing enablement or mask symlinks, cover both /run and /etc 'systemctl disable --runtime' would disable a unit, but only if it was enabled with '--runtime', and silently do nothing if the unit was enabled persistently. And similarly 'systemctl disable' would do nothing if the unit was enabled in /run. This just doesn't seem useful. This pathch changes enable/disable and mask/unmask to be asymmetrical. enable and mask create symlinks in /etc or /run, depending on whether --runtime was specified. disable and unmask remove symlinks from both locations. --runtime cannot be specified for the disable and unmask verbs. The advantage is that 'disable' now means that the unit is disabled, period. And similarly for 'unmask', all masks are removed. Similarly for preset and preset-all, they now cannot be called with --runtime, and are asymmetrical: when they enable a unit, symlinks are created in /etc. When they disable a unit, all symlinks are nuked. $ systemctl --root=/ enable bluetooth Created symlink /etc/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service. Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service. $ systemctl --root=/ --runtime enable bluetooth Created symlink /run/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service. Created symlink /run/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service. $ systemctl --root=/ disable bluetooth Removed /run/systemd/system/bluetooth.target.wants/bluetooth.service. Removed /run/systemd/system/dbus-org.bluez.service. Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service. Removed /etc/systemd/system/dbus-org.bluez.service. $ systemctl --root=/ disable --runtime bluetooth --runtime cannot be used with disable $ systemctl --root=/ mask --runtime bluetooth Created symlink /run/systemd/system/bluetooth.service → /dev/null. $ systemctl --root=/ mask bluetooth Created symlink /etc/systemd/system/bluetooth.service → /dev/null. $ systemctl --root=/ unmask bluetooth Removed /run/systemd/system/bluetooth.service. Removed /etc/systemd/system/bluetooth.service. $ systemctl --root=/ unmask --runtime bluetooth --runtime cannot be used with unmask $ systemctl --root=/ --runtime enable bluetooth Created symlink /run/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service. Created symlink /run/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service. $ systemctl --root=/ enable bluetooth Created symlink /etc/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service. Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service. $ systemctl --root=/ preset bluetooth Removed /run/systemd/system/bluetooth.target.wants/bluetooth.service. Removed /run/systemd/system/dbus-org.bluez.service. Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service. Removed /etc/systemd/system/dbus-org.bluez.service. $ systemctl --root=/ preset --runtime bluetooth --runtime cannot be used with preset $ systemctl preset-all --runtime --runtime cannot be used with preset-all ceph-disk should be updated to adopt the change of systemd.
Is this a problem with the _packaging_ in Fedora? (It doesn't seem like it.) Red Hat Bugzilla is not the place where upstream Ceph developers watch for new bugs. You'll have better luck opening a bz in the Ceph tracker at https://tracker.ceph.com/ If you do that you can add it to the External Trackers up above; then maybe assign it to a Red Hat Ceph developer or close it. Red Hat's bundled packages on RHEL, including RHEL8, are built from downstream sources, and, as Boris notes above, don't include the server components. AFAIK there are no packages of upstream Ceph for RHEL8.
We no longer build ceph-disk in Fedora since the Nautilus release that removed the ceph-disk bits, closing.