DescriptionDeepthi Dharwar
2018-01-24 08:55:39 UTC
Description of problem:
Using the latest ceph-ansible, to deploy Ceph in Containers.
Ceph cluster comprises of 1 MON, 1 MGR and 1 OSD - NVMe.
Currently when we try to deploy the same, the OSD container keeps restarting as it fails to execute entrypoint.sh disk_list.sh.
When we use NVMe, the partitions are /dev/nvme0n1p1 and /dev/nvme0n1p2.
Looks like it is appending 1 to the OSD_DEVICE so it is searching for /dev/nvme0n11 instead, while trying to mount ceph data.
The device is prepared properly but when activating the disk, it fails.
There are no errors on the ceph-ansible side as it has deployed the container.
Version-Release number of selected component (if applicable):
How reproducible: Very easy. Just need to have a single OSD backed to NVMe and run Ceph OSD process for the same in containers.
Steps to Reproduce:
1. Use ceph-ansible to deploy Ceph in Containers comprising of 1 MON, 1 MGR and 1 OSD
2. Make sure the OSD is backed to NVMe device.
3. Once ceph-ansible deploys, the CEPH-OSD container keeps restarting as it fails to execute disk_list script.
Actual results:
CEPH-OSD container keeps restarting as it fails to mount ceph data.
# ./ceph-osd-run.sh nvme0n1
mount: special device /dev/nvme0n11 does not exist
Error response from daemon: No such container: expose_partitions_nvme0n1
2018-01-24 03:35:22 /entrypoint.sh: static: does not generate config
mount: special device /dev/nvme0n11 does not exist
Expected results:
Ceph OSD container to be up and running when backed to NVMe
Additional info:
function mount_ceph_data () {
if is_dmcrypt; then
mount /dev/mapper/"${data_uuid}" "$tmp_dir"
else
if is_loop_dev "${OSD_DEVICE}"; then
mount "${OSD_DEVICE}p1" "$tmp_dir"
else
mount "${OSD_DEVICE}1" "$tmp_dir"
fi
fi
Need a check if the device is NVMe and append 'p1' instead of '1' while mounting the disk.
Upstream bug: https://github.com/ceph/ceph-container/issues/884
QE had tried using ceph-ansible-3.0.28-1.el7cp.noarch and container image - ceph-3.0-rhel-7-docker-candidate-53533-20180320051359
Was working fine.
Regards,
Vasishta Shastry
AQE, Ceph
Description of problem: Using the latest ceph-ansible, to deploy Ceph in Containers. Ceph cluster comprises of 1 MON, 1 MGR and 1 OSD - NVMe. Currently when we try to deploy the same, the OSD container keeps restarting as it fails to execute entrypoint.sh disk_list.sh. When we use NVMe, the partitions are /dev/nvme0n1p1 and /dev/nvme0n1p2. Looks like it is appending 1 to the OSD_DEVICE so it is searching for /dev/nvme0n11 instead, while trying to mount ceph data. The device is prepared properly but when activating the disk, it fails. There are no errors on the ceph-ansible side as it has deployed the container. Version-Release number of selected component (if applicable): How reproducible: Very easy. Just need to have a single OSD backed to NVMe and run Ceph OSD process for the same in containers. Steps to Reproduce: 1. Use ceph-ansible to deploy Ceph in Containers comprising of 1 MON, 1 MGR and 1 OSD 2. Make sure the OSD is backed to NVMe device. 3. Once ceph-ansible deploys, the CEPH-OSD container keeps restarting as it fails to execute disk_list script. Actual results: CEPH-OSD container keeps restarting as it fails to mount ceph data. # ./ceph-osd-run.sh nvme0n1 mount: special device /dev/nvme0n11 does not exist Error response from daemon: No such container: expose_partitions_nvme0n1 2018-01-24 03:35:22 /entrypoint.sh: static: does not generate config mount: special device /dev/nvme0n11 does not exist Expected results: Ceph OSD container to be up and running when backed to NVMe Additional info: function mount_ceph_data () { if is_dmcrypt; then mount /dev/mapper/"${data_uuid}" "$tmp_dir" else if is_loop_dev "${OSD_DEVICE}"; then mount "${OSD_DEVICE}p1" "$tmp_dir" else mount "${OSD_DEVICE}1" "$tmp_dir" fi fi Need a check if the device is NVMe and append 'p1' instead of '1' while mounting the disk. Upstream bug: https://github.com/ceph/ceph-container/issues/884