Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2078008

Summary: [RHOS16.2] encrypted volumes can only be uploaded as raw, but the api lets you try to convert on upload and fail
Product: Red Hat OpenStack Reporter: Brian Rosmaita <brian.rosmaita>
Component: openstack-cinderAssignee: Brian Rosmaita <brian.rosmaita>
Status: CLOSED ERRATA QA Contact: Tzach Shefi <tshefi>
Severity: low Docs Contact: Andy Stillman <astillma>
Priority: low    
Version: 16.0 (Train)CC: bkopilov, eharney, gfidente, ltoscano, tshefi
Target Milestone: z4Keywords: Triaged, ZStream
Target Release: 16.2 (Train on RHEL 8.4)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-cinder-15.6.1-2.20220818134749.f3340ec.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1798890 Environment:
Last Closed: 2022-12-07 19:22:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1798890    

Description Brian Rosmaita 2022-04-22 19:57:43 UTC
+++ This bug was initially created as a clone of Bug #1798890 +++

Description of problem:
RHOS16 with all in one setup (virtual)
Ceph as glance, cinder and nova backend.


How reproducible:
Always

Steps to Reproduce:
#1 create image1 from a file (nova boot from this image works !!)
#2 create encrypted volume1 from image1 
#3 Create image2 with upload-to-image (volume1, image2)
New glance image created (image2).
#4 nova boot from image2 - > Nova report Active but there is no OS inside.

Actual results:
nova is up and active but unable to access to OS (it was not loaded)

Expected results:
instance OS should be up and running.

Additional info:

--- Additional comment from bkopilov on 2020-02-06 09:27:15 UTC ---

More information: the upload-to-image used by default RAW image and as far as i know not supported on ceph,
I tried to convert it to qcow2

/home/heat-admin
==== controller-2 controllers ====
/var/log/containers/cinder/cinder-volume.log:397:2020-02-06 09:15:21.610 13026 ERROR cinder.volume.manager [req-205a21f8-7471-48f9-8c45-eb1e1a830de1 99beb5bee51240bcbd39e74ad53eda96 2b46203ca51e4ff3afc1a6734d663c8a - default default] Upload volume to image encountered an error (image-id: a6c29719-a05b-4cbf-97e5-b0f7b3ff6b36).: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
/var/log/containers/cinder/cinder-volume.log:399:2020-02-06 09:15:21.702 13026 WARNING cinder.volume.manager [req-205a21f8-7471-48f9-8c45-eb1e1a830de1 99beb5bee51240bcbd39e74ad53eda96 2b46203ca51e4ff3afc1a6734d663c8a - default default] Deleting image in unexpected status: queued.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
/var/log/containers/cinder/cinder-volume.log:400:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server [req-205a21f8-7471-48f9-8c45-eb1e1a830de1 99beb5bee51240bcbd39e74ad53eda96 2b46203ca51e4ff3afc1a6734d663c8a - default default] Exception during message handling: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
/var/log/containers/cinder/cinder-volume.log:405:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
/var/log/containers/cinder/cinder-volume.log:406:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
/var/log/containers/cinder/cinder-volume.log:407:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
/var/log/containers/cinder/cinder-volume.log:408:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch
/var/log/containers/cinder/cinder-volume.log:409:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
/var/log/containers/cinder/cinder-volume.log:410:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
/var/log/containers/cinder/cinder-volume.log:411:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
/var/log/containers/cinder/cinder-volume.log:412:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/volume/manager.py", line 1687, in copy_volume_to_image
/var/log/containers/cinder/cinder-volume.log:413:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     payload['message'] = six.text_type(error)
/var/log/containers/cinder/cinder-volume.log:414:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
/var/log/containers/cinder/cinder-volume.log:415:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     self.force_reraise()
/var/log/containers/cinder/cinder-volume.log:416:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
/var/log/containers/cinder/cinder-volume.log:417:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
/var/log/containers/cinder/cinder-volume.log:418:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
/var/log/containers/cinder/cinder-volume.log:419:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     raise value
/var/log/containers/cinder/cinder-volume.log:420:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/volume/manager.py", line 1668, in copy_volume_to_image
/var/log/containers/cinder/cinder-volume.log:421:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     image_service, image_meta)
/var/log/containers/cinder/cinder-volume.log:422:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 1595, in copy_volume_to_image
/var/log/containers/cinder/cinder-volume.log:423:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     image_meta, tmp_file)
/var/log/containers/cinder/cinder-volume.log:424:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/image/image_utils.py", line 700, in upload_volume
/var/log/containers/cinder/cinder-volume.log:425:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     compress=compress)
/var/log/containers/cinder/cinder-volume.log:426:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/image/image_utils.py", line 328, in convert_image
/var/log/containers/cinder/cinder-volume.log:427:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     compress=compress)
/var/log/containers/cinder/cinder-volume.log:428:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/image/image_utils.py", line 271, in _convert_image
/var/log/containers/cinder/cinder-volume.log:429:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     utils.execute(*cmd, run_as_root=run_as_root)
/var/log/containers/cinder/cinder-volume.log:430:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/cinder/utils.py", line 126, in execute
/var/log/containers/cinder/cinder-volume.log:431:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     return processutils.execute(*cmd, **kwargs)
/var/log/containers/cinder/cinder-volume.log:432:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py", line 424, in execute
/var/log/containers/cinder/cinder-volume.log:433:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server     cmd=sanitized_cmd)
/var/log/containers/cinder/cinder-volume.log:434:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
/var/log/containers/cinder/cinder-volume.log:435:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server Command: qemu-img convert -O qcow2 -c /var/lib/cinder/conversion/volume-eb097bfa-b81f-424c-b20a-c040d131aec1-a6c29719-a05b-4cbf-97e5-b0f7b3ff6b36 /var/lib/cinder/conversion/tmp8p1lxuvi
/var/log/containers/cinder/cinder-volume.log:436:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server Exit code: 1
/var/log/containers/cinder/cinder-volume.log:437:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server Stdout: ''
/var/log/containers/cinder/cinder-volume.log:438:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server Stderr: "qemu-img: Could not open '/var/lib/cinder/conversion/volume-eb097bfa-b81f-424c-b20a-c040d131aec1-a6c29719-a05b-4cbf-97e5-b0f7b3ff6b36': Parameter 'key-secret' is required for cipher\n"
/var/log/containers/cinder/cinder-volume.log:439:2020-02-06 09:15:22.770 13026 ERROR oslo_messaging.rpc.server

--- Additional comment from Eric Harney on 2020-02-10 15:20:39 UTC ---

Comment #1 is a whole separate issue from the title/description of this bz.  Can you split it into another BZ?

--- Additional comment from bkopilov on 2020-02-10 15:25:14 UTC ---

(In reply to Eric Harney from comment #2)
> Comment #1 is a whole separate issue from the title/description of this bz. 
> Can you split it into another BZ?

Hi Eric , 
Here is an email:

Brian Rosmaita
Thu, Feb 6, 3:56 PM (4 days ago)
to me, Luigi, Eric, Brian, rhos-cinder, Tzach

Apologies for top-posting.  It think we have 3 different bugs here:

1. booting an instance in Nova from a Glance image created from an
encrypted volume by the Cinder upload-volume-to-image action is *not*
supported by Nova.  The bug is that instead of letting the instance go
to ACTIVE, Nova should return some kind of error (maybe at the Compute
API layer; if the image has the cinder_encryption_key_id metadata on it,
you can't boot from it).

2. trying to upload an encrypted volume as an image as qcow2 is failing
(need to check with Eric about whether this is supported or not).

3. Possible data loss from createImage action -- hopefully this is not
something people are likely to do, but if someone does the unsupported
Nova action described in #1, Benny verified that even though the
instance isn't usable, you can do the 'nova image-create' action on it.
This results in another image (presumable unusable) that has the same
cinder_encryption_key_id and cinder_encryption_key_deletion_policy as
the original image -- so when this useless image is deleted, the key for
the usable image is deleted.

Bug #3 is really bad.  It can be fixed short-term by a configuration
change in Nova, namely, by including the cinder_encryption_key_* in the
Nova non_inheritable_image_properties list.  (Longer term, it would be
fixed by the fix to #1, which wouldn't let this case happen.)

--- Additional comment from Eric Harney on 2020-02-10 15:39:02 UTC ---

Right.  But we can't handle 3 different bugs in the same BZ.  #1 probably needs a doc note since it is expected but not obvious behavior.  #2 needs a Cinder fix.  #3 needs work in Nova etc.

--- Additional comment from RHEL Program Management on 2020-02-10 15:39:09 UTC ---

Since this bug report now has devel_ack+, the Devel Conditional NAK state is no longer valid and has been cleared.

--- Additional comment from Brian Rosmaita on 2020-02-10 16:13:11 UTC ---

I've posted some additional BZs to split this up.  Using the numbers from comment #3:

1. https://bugzilla.redhat.com/show_bug.cgi?id=1801282 -- (nova API change to prevent boot of image created from encrypted cinder volume)

2. this bug (encrypted volumes can only be uploaded as raw, but the api lets you try to convert on upload and fail)

3. https://bugzilla.redhat.com/show_bug.cgi?id=1801255 -- blacklist cinder_encryption_key_* properties so that nova doesn't put them on images)

--- Additional comment from Luigi Toscano on 2020-08-28 15:41:07 UTC ---

Moving to z3 after priority and status check.

--- Additional comment from Luigi Toscano on 2020-10-21 14:32:02 UTC ---

No fix yet, moving to z4.

--- Additional comment from Brian Rosmaita on 2021-07-09 15:17:27 UTC ---

Comment #6 addressed the needinfo for this bug.

Comment 8 Tzach Shefi 2022-09-07 13:40:49 UTC
Verified on:
openstack-cinder-15.6.1-2.20220818134749.f3340ec.el8ost.noarch

On a Ceph backed deployment, lets follow the reproduce steps. 

1. Upload an image to glance from a file:
(overcloud) [stack@undercloud-0 ~]$ glance image-create --disk-format raw --container-format bare --file cirros-0.4.0-x86_64-disk.raw --name cirros.raw
+------------------+----------------------------------------------------------------------------------+
| Property         | Value                                                                            |
+------------------+----------------------------------------------------------------------------------+
| checksum         | ba3cd24377dde5dfdd58728894004abb                                                 |
| container_format | bare                                                                             |
| created_at       | 2022-09-07T10:57:19Z                                                             |
| direct_url       | rbd://6ba3c51e-359d-49db-                                                        |
|                  | af30-ad92b9b037cb/images/839aed48-c6d3-4edd-9e04-05e5f6cbc94c/snap               |
| disk_format      | raw                                                                              |
| id               | 839aed48-c6d3-4edd-9e04-05e5f6cbc94c                                             |
| locations        | [{"url": "rbd://6ba3c51e-359d-49db-                                              |
|                  | af30-ad92b9b037cb/images/839aed48-c6d3-4edd-9e04-05e5f6cbc94c/snap", "metadata": |
|                  | {"store": "default_backend"}}]                                                   |
| min_disk         | 0                                                                                |
| min_ram          | 0                                                                                |
| name             | cirros.raw                                                                       |
| os_hash_algo     | sha512                                                                           |
| os_hash_value    | b795f047a1b10ba0b7c95b43b2a481a59289dc4cf2e49845e60b194a911819d3ada03767bbba4143 |
|                  | b44c93fd7f66c96c5a621e28dff51d1196dae64974ce240e                                 |
| os_hidden        | False                                                                            |
| owner            | 1fadff65f8b74b9aaf2144739f25123b                                                 |
| protected        | False                                                                            |
| size             | 46137344                                                                         |
| status           | active                                                                           |
| stores           | default_backend                                                                  |
| tags             | []                                                                               |
| updated_at       | 2022-09-07T10:57:21Z                                                             |
| virtual_size     | Not available                                                                    |
| visibility       | shared                                                                           |
+------------------+----------------------------------------------------------------------------------+

Lets confirm an instance can bootup from this image:
(overcloud) [stack@undercloud-0 ~]$ nova boot FirstInstance --flavor tiny --image cirros.raw --nic net-id=2e2e6133-8106-46f9-88b0-68a2650809df
+--------------------------------------+---------------------------------------------------+
| Property                             | Value                                             |
+--------------------------------------+---------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                            |
| OS-EXT-AZ:availability_zone          |                                                   |
| OS-EXT-SRV-ATTR:host                 | -                                                 |
| OS-EXT-SRV-ATTR:hostname             | firstinstance                                     |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                 |
| OS-EXT-SRV-ATTR:instance_name        |                                                   |
| OS-EXT-SRV-ATTR:kernel_id            |                                                   |
| OS-EXT-SRV-ATTR:launch_index         | 0                                                 |
| OS-EXT-SRV-ATTR:ramdisk_id           |                                                   |
| OS-EXT-SRV-ATTR:reservation_id       | r-oajt54zg                                        |
| OS-EXT-SRV-ATTR:root_device_name     | -                                                 |
| OS-EXT-SRV-ATTR:user_data            | -                                                 |
| OS-EXT-STS:power_state               | 0                                                 |
| OS-EXT-STS:task_state                | scheduling                                        |
....
..                        |
| tenant_id                            | 1fadff65f8b74b9aaf2144739f25123b                  |
| trusted_image_certificates           | -                                                 |
| updated                              | 2022-09-07T11:02:11Z                              |
| user_id                              | 9d9c74f8fa5644598990efa83cc3f140                  |
+--------------------------------------+---------------------------------------------------+

Yep image is OK and instance is alive:
(overcloud) [stack@undercloud-0 ~]$ nova list
+--------------------------------------+---------------+--------+------------+-------------+------------------------------------------+
| ID                                   | Name          | Status | Task State | Power State | Networks                                 |
+--------------------------------------+---------------+--------+------------+-------------+------------------------------------------+
| c7cee8fd-04b0-4559-9635-8b60fa73cb99 | FirstInstance | ACTIVE | -          | Running     | nova=2620:52:0:13b8::1000:42, 10.0.0.215 |


2. Create an encrypted volume from said image
(overcloud) [stack@undercloud-0 ~]$ cinder create 3 --image cirros.raw --volume-type LUKS --name EncVolFromImage
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2022-09-07T11:06:36.000000           |
| description                    | None                                 |
| encrypted                      | True                                 |
| id                             | 92e21799-3c29-4efc-8a90-ff5356bdf1c6 |
| metadata                       | {}                                   |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | EncVolFromImage                      |
| os-vol-host-attr:host          | None                                 |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | 1fadff65f8b74b9aaf2144739f25123b     |
| replication_status             | None                                 |
| size                           | 3                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | creating                             |
| updated_at                     | None                                 |
| user_id                        | 9d9c74f8fa5644598990efa83cc3f140     |
| volume_type                    | LUKS                                 |
+--------------------------------+--------------------------------------+


(overcloud) [stack@undercloud-0 ~]$ cinder list
+--------------------------------------+-----------+-----------------+------+-----------------+----------+--------------------------------------+
| ID                                   | Status    | Name            | Size | Volume Type     | Bootable | Attached to                          |
+--------------------------------------+-----------+-----------------+------+-----------------+----------+--------------------------------------+
| 92e21799-3c29-4efc-8a90-ff5356bdf1c6 | available | EncVolFromImage | 3    | LUKS            | true     |                                      |


Volume was successfully created. 

3. Create a new image using upload-to-image from the new encrypted volume
(overcloud) [stack@undercloud-0 ~]$ cinder upload-to-image EncVolFromImage ImageFromEncVolume
+---------------------+--------------------------------------+
| Property            | Value                                |
+---------------------+--------------------------------------+
| container_format    | bare                                 |
| disk_format         | raw                                  |
| display_description | None                                 |
| id                  | 92e21799-3c29-4efc-8a90-ff5356bdf1c6 |
| image_id            | e773e926-8f04-44b8-9eda-7816c8bf0a03 |
| image_name          | ImageFromEncVolume                   |
| protected           | False                                |
| size                | 3                                    |
| status              | uploading                            |
| updated_at          | 2022-09-07T11:06:57.000000           |
| visibility          | shared                               |
| volume_type         | LUKS                                 |
+---------------------+--------------------------------------+

(overcloud) [stack@undercloud-0 ~]$ glance image-show e773e926-8f04-44b8-9eda-7816c8bf0a03
+---------------------------------------+----------------------------------------------------------------------------------+
| Property                              | Value                                                                            |
+---------------------------------------+----------------------------------------------------------------------------------+
| checksum                              | a3bf12e4b63e1020fa1b91cad1fb36a7                                                 |
| cinder_encryption_key_deletion_policy | on_image_deletion                                                                |
| cinder_encryption_key_id              | 181be706-b9e5-41b3-9d56-79357c1259ed                                             |
| container_format                      | bare                                                                             |
| created_at                            | 2022-09-07T11:10:05Z                                                             |
| direct_url                            | rbd://6ba3c51e-359d-49db-                                                        |
|                                       | af30-ad92b9b037cb/images/e773e926-8f04-44b8-9eda-7816c8bf0a03/snap               |
| disk_format                           | raw                                                                              |
| id                                    | e773e926-8f04-44b8-9eda-7816c8bf0a03                                             |
| locations                             | [{"url": "rbd://6ba3c51e-359d-49db-                                              |
|                                       | af30-ad92b9b037cb/images/e773e926-8f04-44b8-9eda-7816c8bf0a03/snap", "metadata": |
|                                       | {"store": "default_backend"}}]                                                   |
| min_disk                              | 0                                                                                |
| min_ram                               | 0                                                                                |
| name                                  | ImageFromEncVolume                                                               |
| os_hash_algo                          | sha512                                                                           |
| os_hash_value                         | 45ab5b10f746b753941ccc3814cca524df2626570b130ac723319ed9397a76aeed90d1a4ea8a41f7 |
|                                       | 8f60b5fd4f70ad61abc95214e8b0061148cde59a0cbd18cb                                 |
| os_hidden                             | False                                                                            |
| owner                                 | 1fadff65f8b74b9aaf2144739f25123b                                                 |
| protected                             | False                                                                            |
| signature_verified                    | False                                                                            |
| size                                  | 3221225472                                                                       |
| status                                | active                                                                           |
| stores                                | default_backend                                                                  |
| tags                                  | []                                                                               |
| updated_at                            | 2022-09-07T11:10:40Z                                                             |
| virtual_size                          | Not available                                                                    |
| visibility                            | shared                                                                           |
+---------------------------------------+----------------------------------------------------------------------------------+

We proved that an encrypted volume was successfully uploaded to Glance,
when using the default allowed raw/bare combination, 
as they are defaults I didn't even mention them during upload to image. 

Now lets try uploading to Glance as qcow2/bare, it should fail as it isn't supported. 

(overcloud) [stack@undercloud-0 ~]$ cinder upload-to-image EncVolFromImage ImageFromEncVolumeTake2 --disk-format qcow2 --container-format bare 
ERROR: An encrypted volume uploaded as an image must use 'raw' disk_format and 'bare' container_format, which are the defaults for these options. (HTTP 400) (Request-ID: req-1025dc45-18c9-445c-94f9-8537b3b76524)

We failed as expected and got a meaningful explanation as to why, good to verify.

Comment 9 Brian Rosmaita 2022-09-07 14:15:22 UTC
Test LGTM.  Thanks, Tzach!

Comment 16 errata-xmlrpc 2022-12-07 19:22:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.4), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8794