Created attachment 1649489 [details] all.yml provided as input to ceph-ansible 1. Description of problem: I deployed Ceph for OpenStack as usual but I added a new key to the keys list like this: - caps: mgr: allow * mon: profile rbd osd: profile rbd pool=images key: QAYAg9eAAAAABAABL4f6ig8N8cUKCP3Z0PkCQP== mode: '0600' name: client.glance When the deployment all of the keys defined in all.yml were created except the one for client.glance. The call to the following module: https://github.com/ceph/ceph-ansible/blob/v4.0.5/library/ceph_key.py by the TASK [ceph-osd : generate keys] failed with: msg: path /etc/ceph/central.client.glance.keyring does not exist However, the keyring is not yet expected to exist. I'm expecting the module to create the keyring. It creates the other keys defined in all.yaml I'll attach my all.yml 2. Version-Release number of selected component (if applicable): ceph-ansible-4.0.5-1 3. How reproducible: deterministic 4. Steps to Reproduce: Deploy ceph using the attached all.yml 5. Actual results: a. The playbook fails b. All keys are created on the first Mon's file system except the one on the list which made the deployment fail [root@control-plane-controller-0 ~]# ls /etc/ceph/ | sort central.client.admin.keyring central.client.manila.keyring central.client.openstack.keyring central.client.radosgw.keyring central.conf central.mgr.control-plane-controller-0.keyring central.mon.keyring [root@control-plane-controller-0 ~]# c. All keys are created in Ceph except the one on the list which made the deployment fail [root@control-plane-controller-0 ~]# podman exec ceph-mon-$HOSTNAME ceph --cluster central auth list osd.0 key: AQAeig9esUbTGhAABm9sBD55OBt+QR2OdUaGWA== caps: [mgr] allow profile osd caps: [mon] allow profile osd caps: [osd] allow * client.admin key: AQBRgQ9eAAAAABAA1qEjGnX4RJXm6GDz1U4idA== caps: [mds] allow * caps: [mgr] allow * caps: [mon] allow * caps: [osd] allow * client.bootstrap-mds key: AQDIiQ9eX4oqKhAAujX4c9laTmiNxxb9vAtHgQ== caps: [mon] allow profile bootstrap-mds client.bootstrap-mgr key: AQDIiQ9e6qAqKhAAbOvgF4+WJ0aMMorjaArghw== caps: [mon] allow profile bootstrap-mgr client.bootstrap-osd key: AQDIiQ9e/bYqKhAAsAwvoh2CSMTtPoqREtgGMQ== caps: [mon] allow profile bootstrap-osd client.bootstrap-rbd key: AQDIiQ9eD80qKhAA5Z3JXZ6EVQQx/ToSCbwILA== caps: [mon] allow profile bootstrap-rbd client.bootstrap-rbd-mirror key: AQDIiQ9eleIqKhAAyLwS+xyki3YsJu49AD+LXQ== caps: [mon] allow profile bootstrap-rbd-mirror client.bootstrap-rgw key: AQDIiQ9ehvcqKhAADscyd02vCW84lWGXzrALIg== caps: [mon] allow profile bootstrap-rgw client.manila key: AQBRgQ9eAAAAABAAv84zEilJYZPNuJ0Iwn9Ndg== caps: [mds] allow * caps: [mgr] allow * caps: [mon] allow r, allow command 'auth del', allow command 'auth caps', allow command 'auth get', allow command 'auth get-or-create' caps: [osd] allow rw client.openstack key: AQBRgQ9eAAAAABAAB/7pyyCPh0ToEdiBtxGdLg== caps: [mgr] allow * caps: [mon] profile rbd caps: [osd] profile rbd pool=volumes, profile rbd pool=backups, profile rbd pool=vms, profile rbd pool=images mgr.control-plane-controller-0 key: AQDpiQ9eAAAAABAA2rXTcpWGs3TbhPIvSrG6WQ== caps: [mds] allow * caps: [mon] allow profile mgr caps: [osd] allow * installed auth entries: [root@control-plane-controller-0 ~]# 6. Expected results: Ceph is deployed without error and the additional keyring can be seen after running `podman exec ceph-mon-$HOSTNAME ceph --cluster central auth list` and a keyring file is created on the mon node called /etc/ceph/central.client.glance.keyring 7. Additional info: Issue can also be seen in TripleO CI: https://review.opendev.org/#/c/700947/ under scenario 001 standalone
Please specify the severity of this bug. Severity is defined here: https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.
Created attachment 1649490 [details] last 500 lines of ceph-ansible log
Created attachment 1649491 [details] tarball of full ceph-ansible env
Created attachment 1649492 [details] tarball of full ceph-ansible env
Created attachment 1649493 [details] full ceph-ansible log
It seems to be user error on my part. I updated my input's key to the following and it worked [1]. key = AQBRgQ9eAAAAABAAv84zEilJYZPNuJ0Iwn9Ndg== The original key I passed it couldn't be decoded: stderr: can't decode key 'QAYAg9eAAAAABAABL4f6ig8N8cUKCP3Z0PkCQP==' [1] File created: [root@control-plane-controller-0 ~]# cat /etc/ceph/central.client.glance.keyring [client.glance] key = AQBRgQ9eAAAAABAAv84zEilJYZPNuJ0Iwn9Ndg== caps mgr = "allow *" caps mon = "profile rbd" caps osd = "profile rbd pool=images" [root@control-plane-controller-0 ~]# Entry found in ceph auth list [root@control-plane-controller-0 ~]# podman exec ceph-mon-$HOSTNAME ceph --cluster central auth list osd.0 key: AQAeig9esUbTGhAABm9sBD55OBt+QR2OdUaGWA== caps: [mgr] allow profile osd caps: [mon] allow profile osd caps: [osd] allow * client.admin key: AQBRgQ9eAAAAABAA1qEjGnX4RJXm6GDz1U4idA== caps: [mds] allow * caps: [mgr] allow * caps: [mon] allow * caps: [osd] allow * client.bootstrap-mds key: AQDIiQ9eX4oqKhAAujX4c9laTmiNxxb9vAtHgQ== caps: [mon] allow profile bootstrap-mds client.bootstrap-mgr key: AQDIiQ9e6qAqKhAAbOvgF4+WJ0aMMorjaArghw== caps: [mon] allow profile bootstrap-mgr client.bootstrap-osd key: AQDIiQ9e/bYqKhAAsAwvoh2CSMTtPoqREtgGMQ== caps: [mon] allow profile bootstrap-osd client.bootstrap-rbd key: AQDIiQ9eD80qKhAA5Z3JXZ6EVQQx/ToSCbwILA== caps: [mon] allow profile bootstrap-rbd client.bootstrap-rbd-mirror key: AQDIiQ9eleIqKhAAyLwS+xyki3YsJu49AD+LXQ== caps: [mon] allow profile bootstrap-rbd-mirror client.bootstrap-rgw key: AQDIiQ9ehvcqKhAADscyd02vCW84lWGXzrALIg== caps: [mon] allow profile bootstrap-rgw client.glance key: AQBRgQ9eAAAAABAAv84zEilJYZPNuJ0Iwn9Ndg== caps: [mgr] allow * caps: [mon] profile rbd caps: [osd] profile rbd pool=images client.manila key: AQBRgQ9eAAAAABAAv84zEilJYZPNuJ0Iwn9Ndg== caps: [mds] allow * caps: [mgr] allow * caps: [mon] allow r, allow command 'auth del', allow command 'auth caps', allow command 'auth get', allow command 'auth get-or-create' caps: [osd] allow rw