.Deploying encrypted OSDs in containers by using `ceph-disk` works as expected
When attempting to deploy a containerized OSD by using the `ceph-disk` and `dmcrypt` utilities, the container process failed to start because the OSD ID could not be found by the mounts table. With this update, the OSD ID is correctly determined, and the container process no longer fails.
DescriptionDimitri Savineau
2019-04-03 19:35:48 UTC
Description of problem:
When trying to deploy OSD using ceph-disk and dmcrypt (either bluestore or filestore), the container process will fail to start because the OSD ID can't be found via the mounts table.
The OSD id is determined by looking the dmcrypt partition in /proc/mounts but we're looking for /dev/dm-X instead of the /dev/mapper/{UUID} due to the readlink -f command.
Version-Release number of selected component (if applicable):
RHCS 3.2 container image rhceph-3-rhel7:3-23
ceph version 12.2.8-89.el7cp (2f66ab2fa63b2879913db6d6cf314572a83fd1f0) luminous (stable)
How reproducible:
100%
Steps to Reproduce:
1. Deploy OSD with dmcrypt enabled (-e OSD_DMCRYPT=1) and ceph-disk (-e CEPH_DAEMON=OSD_CEPH_DISK_ACTIVATE)
ceph-ansible config used:
-------
osd_scenario: collocated
dmcrypt: true
devices:
- /dev/sdb
- /dev/sdc
- /dev/sdd
- /dev/sde
-------
Actual results:
the container fails to start due to OSD ID variable empty on the ceph-osd process
: exec: PID 29737: spawning /usr/bin/ceph-osd --cluster ceph -f -i --setuser ceph --setgroup disk
: exec: Waiting 29737 to quit
(...)
: teardown: managing teardown after SIGCHLD
: teardown: Waiting PID 29737 to terminate
: teardown: Process 29737 is terminated
: teardown: Bye Bye, container will die with return code -1
: teardown: if you don't want me to die and have access to a shell to debug this situation, next time run me with '-e DEBUG=stayalive'
Expected results:
the container should start.
: exec: PID 31469: spawning /usr/bin/ceph-osd --cluster ceph -f -i 0 --setuser ceph --setgroup disk
: exec: Waiting 31469 to quit
: starting osd.0 at - osd_data /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2019:0912
Description of problem: When trying to deploy OSD using ceph-disk and dmcrypt (either bluestore or filestore), the container process will fail to start because the OSD ID can't be found via the mounts table. The OSD id is determined by looking the dmcrypt partition in /proc/mounts but we're looking for /dev/dm-X instead of the /dev/mapper/{UUID} due to the readlink -f command. Version-Release number of selected component (if applicable): RHCS 3.2 container image rhceph-3-rhel7:3-23 ceph version 12.2.8-89.el7cp (2f66ab2fa63b2879913db6d6cf314572a83fd1f0) luminous (stable) How reproducible: 100% Steps to Reproduce: 1. Deploy OSD with dmcrypt enabled (-e OSD_DMCRYPT=1) and ceph-disk (-e CEPH_DAEMON=OSD_CEPH_DISK_ACTIVATE) ceph-ansible config used: ------- osd_scenario: collocated dmcrypt: true devices: - /dev/sdb - /dev/sdc - /dev/sdd - /dev/sde ------- Actual results: the container fails to start due to OSD ID variable empty on the ceph-osd process : exec: PID 29737: spawning /usr/bin/ceph-osd --cluster ceph -f -i --setuser ceph --setgroup disk : exec: Waiting 29737 to quit (...) : teardown: managing teardown after SIGCHLD : teardown: Waiting PID 29737 to terminate : teardown: Process 29737 is terminated : teardown: Bye Bye, container will die with return code -1 : teardown: if you don't want me to die and have access to a shell to debug this situation, next time run me with '-e DEBUG=stayalive' Expected results: the container should start. : exec: PID 31469: spawning /usr/bin/ceph-osd --cluster ceph -f -i 0 --setuser ceph --setgroup disk : exec: Waiting 31469 to quit : starting osd.0 at - osd_data /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal