Description of problem:
As subject
Version-Release number of selected component (if applicable):
systemd-239-6.git9f3aed1.fc29.x86_64
ceph-base-12.2.9-1.fc29.x86_64
ceph-deploy-1.5.32-8.fc29.noarch
How reproducible:
100%
Steps to Reproduce:
1. Prepare a ceph cluseter as following steps:
i. # ceph-deploy new --public-network=NETWORK --cluster-network=NETWORK MON
ii. # ceph-deploy install admin MON OSD1 OSD2
iii. # ceph-deploy mon create-initial
iv. Prepare /dev/vdb1 as OSDs in host OSD1 OSD2
v. # ceph-deploy osd create OSD1:/dev/vdb1 OSD2:/dev/vdb1
vi. # ceph-deploy osd activate OSD1:/dev/vdb1 OSD2:/dev/vdb1
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] Traceback (most recent call last):
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/sbin/ceph-disk", line 11, in <module>
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5736, in run
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] main(sys.argv[1:])
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5687, in main
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] args.func(args)
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3829, in main_activate
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] osd_id=osd_id,
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3376, in start_daemon
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] raise Error('ceph osd start failed', e)
[vm-74-208.lab.eng.pek2.redhat.com][WARNIN] ceph_disk.main.Error: Error: ceph osd start failed: Command '['/usr/bin/systemctl', 'disable', 'ceph-osd@0', '--runtime']' retur
ned non-zero exit status 1
[vm-74-208.lab.eng.pek2.redhat.com][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /dev/vdb1
2. Manually run ceph-disk command on OSD1 host:
# # /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /dev/vdb1
main_activate: path = /dev/vdb1
get_dm_uuid: get_dm_uuid /dev/vdb1 uuid path is /sys/dev/block/252:17/dm/uuid
command: Running command: /usr/sbin/blkid -o udev -p /dev/vdb1
command: Running command: /sbin/blkid -p -s TYPE -o value -- /dev/vdb1
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
mount: Mounting /dev/vdb1 on /var/lib/ceph/tmp/mnt.HUgGTT with options noatime,inode64
command_check_call: Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/vdb1 /var/lib/ceph/tmp/mnt.HUgGTT
command: Running command: /usr/sbin/restorecon /var/lib/ceph/tmp/mnt.HUgGTT
activate: Cluster uuid is bdc9c6d8-e2f7-43af-817e-0e0b43b626fc
command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
activate: Cluster name is ceph
activate: OSD uuid is cc96300c-e269-430b-bc9c-6caab79535c2
activate: OSD id is 0
activate: Marking with init system systemd
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/tmp/mnt.HUgGTT/systemd
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.HUgGTT/systemd
activate: ceph osd.0 data dir is ready at /var/lib/ceph/tmp/mnt.HUgGTT
mount_activate: ceph osd.0 already mounted in position; unmounting ours.
unmount: Unmounting /var/lib/ceph/tmp/mnt.HUgGTT
command_check_call: Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.HUgGTT
start_daemon: Starting ceph osd.0...
command_check_call: Running command: /usr/bin/systemctl disable ceph-osd@0
command_check_call: Running command: /usr/bin/systemctl disable ceph-osd@0 --runtime
--runtime cannot be used with disable
Traceback (most recent call last):
File "/usr/sbin/ceph-disk", line 11, in <module>
load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5736, in run
main(sys.argv[1:])
File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 5687, in main
args.func(args)
File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3829, in main_activate
osd_id=osd_id,
File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3376, in start_daemon
raise Error('ceph osd start failed', e)
ceph_disk.main.Error: Error: ceph osd start failed: Command '['/usr/bin/systemctl', 'disable', 'ceph-osd@0', '--runtime']' returned non-zero exit status 1
Actual results:
As above
Expected results:
ceph-disk cmd runs well.
Additional info:
This error happened because systemd doesn't support disable verb with --runtime option since systemd-239. Here is the code commit of this change:
commit 4910b35078ad24dcbc63f372b2fee087640201d0
Author: Zbigniew Jędrzejewski-Szmek <zbyszek.pl>
Date: Thu May 31 15:28:07 2018 +0200
systemctl: when removing enablement or mask symlinks, cover both /run and /etc
'systemctl disable --runtime' would disable a unit, but only if it was enabled
with '--runtime', and silently do nothing if the unit was enabled persistently.
And similarly 'systemctl disable' would do nothing if the unit was enabled in
/run. This just doesn't seem useful.
This pathch changes enable/disable and mask/unmask to be asymmetrical. enable
and mask create symlinks in /etc or /run, depending on whether --runtime was
specified. disable and unmask remove symlinks from both locations. --runtime
cannot be specified for the disable and unmask verbs.
The advantage is that 'disable' now means that the unit is disabled, period.
And similarly for 'unmask', all masks are removed.
Similarly for preset and preset-all, they now cannot be called with --runtime,
and are asymmetrical: when they enable a unit, symlinks are created in /etc.
When they disable a unit, all symlinks are nuked.
$ systemctl --root=/ enable bluetooth
Created symlink /etc/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ --runtime enable bluetooth
Created symlink /run/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /run/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ disable bluetooth
Removed /run/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /run/systemd/system/dbus-org.bluez.service.
Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /etc/systemd/system/dbus-org.bluez.service.
$ systemctl --root=/ disable --runtime bluetooth
--runtime cannot be used with disable
$ systemctl --root=/ mask --runtime bluetooth
Created symlink /run/systemd/system/bluetooth.service → /dev/null.
$ systemctl --root=/ mask bluetooth
Created symlink /etc/systemd/system/bluetooth.service → /dev/null.
$ systemctl --root=/ unmask bluetooth
Removed /run/systemd/system/bluetooth.service.
Removed /etc/systemd/system/bluetooth.service.
$ systemctl --root=/ unmask --runtime bluetooth
--runtime cannot be used with unmask
$ systemctl --root=/ --runtime enable bluetooth
Created symlink /run/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /run/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ enable bluetooth
Created symlink /etc/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ preset bluetooth
Removed /run/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /run/systemd/system/dbus-org.bluez.service.
Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /etc/systemd/system/dbus-org.bluez.service.
$ systemctl --root=/ preset --runtime bluetooth
--runtime cannot be used with preset
$ systemctl preset-all --runtime
--runtime cannot be used with preset-all
ceph-disk should be updated to adopt the change of systemd.
Is this a problem with the _packaging_ in Fedora? (It doesn't seem like it.)
Red Hat Bugzilla is not the place where upstream Ceph developers watch for new bugs. You'll have better luck opening a bz in the Ceph tracker at https://tracker.ceph.com/ If you do that you can add it to the External Trackers up above; then maybe assign it to a Red Hat Ceph developer or close it.
Red Hat's bundled packages on RHEL, including RHEL8, are built from downstream sources, and, as Boris notes above, don't include the server components. AFAIK there are no packages of upstream Ceph for RHEL8.