Bug 1896803 - [cee/sd][ceph-volume] when running playbook add-osd.yml or site.yml ceph-volume does not create OSDs on new devices
Summary: [cee/sd][ceph-volume] when running playbook add-osd.yml or site.yml ceph-volu...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Ceph-Volume
Version: 4.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.3
Assignee: Guillaume Abrioux
QA Contact: Ameena Suhani S H
Ranjini M N
URL:
Whiteboard:
: 1966934 1970031 (view as bug list)
Depends On: 1966934
Blocks: 1890121 2031070
TreeView+ depends on / blocked
 
Reported: 2020-11-11 15:11 UTC by Tomas Petr
Modified: 2022-05-05 07:53 UTC (History)
28 users (show)

Fixed In Version: ceph-14.2.21-8.el8cp, ceph-14.2.21-8.el7cp
Doc Type: Bug Fix
Doc Text:
.The `add-osd.yml` playbook does not fail anymore while creating new OSDs Previously, the `add-osd.yml` playbook would fail when new OSDs were added using `ceph-ansible`. This was due to the `ceph-volume lvm batch` limitation which does not allow addition of new OSDs in a non-interactive mode. With this release, the `--yes` and `--report` options are not passed to the command-line interface and the `add-osd.yml` playbook works as expected when creating new OSDs.
Clone Of:
Environment:
Last Closed: 2022-05-05 07:53:21 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github ceph ceph-ansible pull 6256 0 None Merged library: do not always add --yes in batch mode 2021-11-16 08:46:04 UTC
Red Hat Issue Tracker RHCEPH-1684 0 None None None 2021-09-14 04:39:57 UTC
Red Hat Product Errata RHSA-2022:1716 0 None None None 2022-05-05 07:53:37 UTC

Internal Links: 1970031

Description Tomas Petr 2020-11-11 15:11:53 UTC
Description of problem:
On existing ceph node are deployed OSDs using ceph-ansible ceph-volume batch scenario.
When adding additional devices to the host_vars/node configuration file and running add-osd.yml or site.yml, the new devices are not being populated by OSDs:

Initial node deployment:
----
# cat /etc/ansible/host_vars/osds-3
devices:
  - /dev/vde

dedicated_devices:
  - /dev/vdb

[root@osds-3 ~]# lsblk
NAME                                                                                                                  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda                                                                                                                   253:0    0   20G  0 disk 
├─vda1                                                                                                                253:1    0    1M  0 part 
├─vda2                                                                                                                253:2    0  100M  0 part /boot/efi
└─vda3                                                                                                                253:3    0 19.9G  0 part /
vdb                                                                                                                   253:16   0   30G  0 disk 
└─ceph--block--dbs--199c644a--8657--46f3--8136--b6679cd68153-osd--block--db--7d8b9f07--e822--43da--866b--dc202c3740bd 252:1    0   30G  0 lvm  
vdc                                                                                                                   253:32   0   30G  0 disk 
vdd                                                                                                                   253:48   0   30G  0 disk 
vde                                                                                                                   253:64   0   30G  0 disk 
└─ceph--block--6171157b--47dc--426a--ba3f--5876bb49e42b-osd--block--4b812472--826e--41c4--ba00--c6c0eedc64ae          252:0    0   30G  0 lvm  


Adding new OSDs
----
# cat /etc/ansible/host_vars/osds-3
devices:
  - /dev/vde
  - /dev/vdd

dedicated_devices:
  - /dev/vdb
  - /dev/vdc


# ansible-playbook infrastructure-playbooks/add-osd.yml --limit osds-3
...
2020-11-11 03:15:58,084 p=47055 u=root |  PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
2020-11-11 03:15:58,085 p=47055 u=root |  osds-3                     : ok=184  changed=8    unreachable=0    failed=0    skipped=299  rescued=0    ignored=1   
2020-11-11 03:15:58,093 p=47055 u=root |  Wednesday 11 November 2020  03:15:58 -0500 (0:00:00.056)       0:04:41.749 **** 
2020-11-11 03:15:58,094 p=47055 u=root |  =============================================================================== 
....

[root@osds-3 ~]# lsblk
NAME                                                                                                                  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda                                                                                                                   253:0    0   20G  0 disk 
├─vda1                                                                                                                253:1    0    1M  0 part 
├─vda2                                                                                                                253:2    0  100M  0 part /boot/efi
└─vda3                                                                                                                253:3    0 19.9G  0 part /
vdb                                                                                                                   253:16   0   30G  0 disk 
└─ceph--block--dbs--199c644a--8657--46f3--8136--b6679cd68153-osd--block--db--7d8b9f07--e822--43da--866b--dc202c3740bd 252:1    0   30G  0 lvm  
vdc                                                                                                                   253:32   0   30G  0 disk 
vdd                                                                                                                   253:48   0   30G  0 disk 
vde                                                                                                                   253:64   0   30G  0 disk 
└─ceph--block--6171157b--47dc--426a--ba3f--5876bb49e42b-osd--block--4b812472--826e--41c4--ba00--c6c0eedc64ae          252:0    0   30G  0 lvm  
----

The playbook ends with OK, but no new devices are being deployed.

The task
2020-11-11 03:03:36,685 p=45877 u=root |  TASK [ceph-config : run 'ceph-volume lvm batch --report' to see how many osds are to be created] 

does not set any devices to be deployed


When we have in  host_vars/node configuration file only new devices, the OSDs will get deployed, but that brings different issue, that the 'osd memory target' value is calculated per number of OSDs in host_vars/node configuration file, so it will get calculated wrongly for the amount of OSDs the node will have in total, old+new.


Version-Release number of selected component (if applicable):
ansible-2.8.16-1.el8ae.noarch
ceph-ansible-4.0.31-1.el8cp.noarch
"ceph version 14.2.8-111.el8cp

How reproducible:
Always

Steps to Reproduce:
1. deploy node with x amount of devices 
2. add additional device into  host_vars/node configuration file, keep the old ones
3. try to deploy OSDs on those 

Actual results:


Expected results:


Additional info:

Comment 29 Vasishta 2021-06-09 14:07:41 UTC
*** Bug 1966934 has been marked as a duplicate of this bug. ***

Comment 35 Anjana Suparna Sriram 2021-06-15 10:20:57 UTC
*** Bug 1970031 has been marked as a duplicate of this bug. ***

Comment 47 errata-xmlrpc 2022-05-05 07:53:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat Ceph Storage 4.3 Security and Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1716


Note You need to log in before you can comment on or make changes to this bug.