Description of problem: Openstack image save command fails with the error: $ openstack image save test-rhel8_3 --file download-image.img [Errno 32] Corrupt image download. Hash was e8f732c3fdbb9648e8d68fe534babb6cd0aba52ff524e9f8511a6fa062cc28ac74de379739be7819752f92370c8657f2c2360c0215c5492de8bd80ac47baf1a2 expected 6a90aafc2dcbc7edd8b9f4312d40099dd88c9be2434b8227f0ce9a7a3f4f08ee162ffb256bd372aef340812510fb84f9430c14485bb9daed74afd5f29844e2cc Due to this, a volume from image cannot be created because we get this error ~~~ cinder.exception.ImageDownloadFailed: Failed to download image 72692c04-6aa6-4f08-9353-b7dc9a237e99, reason: IOError: 32 Corrupt image download. Hash was dc85ac2177d5ef863be381e68736caae84e1f6b7c2d25de404172e54f8531ea1070810402533d4aa2a4e6fccf7fe52693d9b9007abe9a09895a40ff05f0b7476 expected 9c623218a59799bc6f6b200028122d84fbfd10995a1f56338420ac0607a091e6238d16a8d9e67c8a2f05ea61c1bec7ae45c3164a6adf8606055de862dc1b2620\n'] ~~~ Then, a fresh image was downloaded from CDN and yet the issue persists. (overcloud) [stack@a0410psecdirector sos]$ openstack image create rhel8_3 --disk-format qcow2 --container-format bare --public --file rhel-8.3-update-2-x86_64-kvm.qcow2 $ openstack image show rhel8_3 -f json { "checksum": "2d3300660874b9986a93657f9346cc29", "container_format": "bare", "created_at": "2021-03-08T06:15:43Z", "disk_format": "qcow2", "file": "/v2/images/ae7809d0-674c-488a-b6a3-e24fcae79591/file", "id": "ae7809d0-674c-488a-b6a3-e24fcae79591", "min_disk": 0, "min_ram": 0, "name": "rhel8_3", "owner": "aa5c111ea4d34431b4dd1f15f8e9be2a", "properties": { "os_hidden": false, "os_hash_algo": "sha512", "os_hash_value": "6a90aafc2dcbc7edd8b9f4312d40099dd88c9be2434b8227f0ce9a7a3f4f08ee162ffb256bd372aef340812510fb84f9430c14485bb9daed74afd5f29844e2cc", "direct_url": "cinder://1ff92640-e960-4bb7-a4b1-e990194e739a", "stores": "default_backend" }, "protected": false, "schema": "/v2/schemas/image", "size": 1305018368, "status": "active", "tags": [], "updated_at": "2021-03-08T06:16:20Z", "virtual_size": null, "visibility": "public" } The error appears to be affecting RHEL images only. Cirros and Fedora images can be downloaded without any issues and volume from these images are created without any issues. Version-Release number of selected component (if applicable): Red Hat OpenStack Platform release 16.1.3 GA (Train) How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: openstack image save command fails with corrupt hash error for RHEL images. Expected results: Openstack image save command should download any image without issues. Additional info: $ cat installed-rpms | grep glance puppet-glance-15.4.1-0.20200611153438.8ac5b13.el8ost.noarch Wed Dec 2 22:33:17 2020 python3-glanceclient-2.17.1-1.20200812213401.1aba8f2.el8ost.noarch Wed Dec 2 22:28:36 2020 Glance backend is cinder Cinder backend is HPE3Par
The root cause issue is now being tracked in https://bugzilla.redhat.com/1940548 and that one will be used for hotfix and delivery.
According to our records, this should be resolved by python-os-brick-2.10.5-1.20220112193419.634fb4a.el8ost. This build is available now.
Verified on: python3-os-brick-2.10.5-1.20220112193419.634fb4a.el8ost.noarch Deployed a system with Glance over Cinder, Where Cinder’s backend is HPE3par FC storage. Lets start with a rhel 7.9 image: (overcloud) [stack@seal41 ~]$ openstack image create --public --disk-format qcow2 --container-format bare --file rhel-server-7.9-update-9-x86_64-kvm.qcow2 rhel-7.9 +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | 82dea14f2fac8b27009c5f926f885f5a | | container_format | bare | | created_at | 2022-04-20T11:34:28Z | | disk_format | qcow2 | | file | /v2/images/aeabc9b4-665f-4ab7-9b47-acda24eca024/file | | id | aeabc9b4-665f-4ab7-9b47-acda24eca024 | | min_disk | 0 | | min_ram | 0 | | name | rhel-7.9 | | owner | 414eab17c9fa4d47be286609ee377514 | | properties | direct_url='cinder://5d0031bc-af3c-4b16-9c6a-f3157808efbd', os_hash_algo='sha512', os_hash_value='5076869a5a0bed649fbfcb3373d1d85090bc315c1c9ccf714307dbe20b2b52f6af9ebc1657f0d3c02dd11775966d67165f3530a8fb176b0ddfccc850059fbb31', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 845833216 | | status | active | | tags | | | updated_at | 2022-04-20T11:34:47Z | | virtual_size | None | | visibility | public | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@seal41 ~]$ openstack image show rhel-7.9 -f json | grep os_hash_value "os_hash_value": "5076869a5a0bed649fbfcb3373d1d85090bc315c1c9ccf714307dbe20b2b52f6af9ebc1657f0d3c02dd11775966d67165f3530a8fb176b0ddfccc850059fbb31", (overcloud) [stack@seal41 ~]$ sha512sum rhel-server-7.9-update-9-x86_64-kvm.qcow2 5076869a5a0bed649fbfcb3373d1d85090bc315c1c9ccf714307dbe20b2b52f6af9ebc1657f0d3c02dd11775966d67165f3530a8fb176b0ddfccc850059fbb31 rhel-server-7.9-update-9-x86_64-kvm.qcow2 (overcloud) [stack@seal41 ~]$ openstack image save rhel-7.9 --file rhel-7.9-downloaded.qcow2 (overcloud) [stack@seal41 ~]$ sha512sum rhel-7.9-downloaded.qcow2 5076869a5a0bed649fbfcb3373d1d85090bc315c1c9ccf714307dbe20b2b52f6af9ebc1657f0d3c02dd11775966d67165f3530a8fb176b0ddfccc850059fbb31 rhel-7.9-downloaded.qcow2 As seen above the sha512 value remains the same for source file, uploaded image and the downloaded image as well, looks good lets replicate for the other images. Now using a rhel 8.2 image: (overcloud) [stack@seal41 ~]$ openstack image create --public --disk-format qcow2 --container-format bare --file rhel-8.2-x86_64-kvm.qcow2 rhel-8.2 +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | b5e819ce2474a99ff569e06b1844b081 | | container_format | bare | | created_at | 2022-04-20T11:40:12Z | | disk_format | qcow2 | | file | /v2/images/07164304-5061-4ca9-a3a9-caf01c5b1023/file | | id | 07164304-5061-4ca9-a3a9-caf01c5b1023 | | min_disk | 0 | | min_ram | 0 | | name | rhel-8.2 | | owner | 414eab17c9fa4d47be286609ee377514 | | properties | direct_url='cinder://aab492f4-d23d-4343-9846-9452dd2c26c8', os_hash_algo='sha512', os_hash_value='1f3cf0cb62931cb13b509da1ee044c1aa07506eaf0dfb77bb673b6d3f56a38e1a3359475611437d82c4fd18c60805b22ae08ab4146ac86cbbd018e4273066bb3', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 1159135232 | | status | active | | tags | | | updated_at | 2022-04-20T11:40:40Z | | virtual_size | None | | visibility | public | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@seal41 ~]$ sha512sum rhel-8.2-x86_64-kvm.qcow2 1f3cf0cb62931cb13b509da1ee044c1aa07506eaf0dfb77bb673b6d3f56a38e1a3359475611437d82c4fd18c60805b22ae08ab4146ac86cbbd018e4273066bb3 rhel-8.2-x86_64-kvm.qcow2 (overcloud) [stack@seal41 ~]$ openstack image save rhel-8.2 --file rhel-8.2-downloaded.qcow2 (overcloud) [stack@seal41 ~]$ sha512sum rhel-8.2-downloaded.qcow2 1f3cf0cb62931cb13b509da1ee044c1aa07506eaf0dfb77bb673b6d3f56a38e1a3359475611437d82c4fd18c60805b22ae08ab4146ac86cbbd018e4273066bb3 rhel-8.2-downloaded.qcow2 Again for rhel 8.2 we get the same sha512 has for source file image and downloaded copy. Lets try rhel 8.3 (overcloud) [stack@seal41 ~]$ openstack image create --public --disk-format qcow2 --container-format bare --file rhel-8.3-x86_64-kvm.qcow2 rhel-8.3 +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | dd554c059e0910379fff88f677f4a4b3 | | container_format | bare | | created_at | 2022-04-20T11:44:06Z | | disk_format | qcow2 | | file | /v2/images/32fd077b-b54e-43ab-a667-87a2a101658a/file | | id | 32fd077b-b54e-43ab-a667-87a2a101658a | | min_disk | 0 | | min_ram | 0 | | name | rhel-8.3 | | owner | 414eab17c9fa4d47be286609ee377514 | | properties | direct_url='cinder://b2d3dfcc-6abd-4934-b312-da17d1c0b5f4', os_hash_algo='sha512', os_hash_value='6acf19b94aaa6b4d9dfbddd2ec5ec07661f1ca4de8a3527e8ec03b20a51ca28e8faedcd22c78ef3c0bac4c0de8f6dc36356d0a589c699058421bff08c2cd4ac6', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 1316683776 | | status | active | | tags | | | updated_at | 2022-04-20T11:44:37Z | | virtual_size | None | | visibility | public | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@seal41 ~]$ sha512sum rhel-8.3-x86_64-kvm.qcow2 6acf19b94aaa6b4d9dfbddd2ec5ec07661f1ca4de8a3527e8ec03b20a51ca28e8faedcd22c78ef3c0bac4c0de8f6dc36356d0a589c699058421bff08c2cd4ac6 rhel-8.3-x86_64-kvm.qcow2 (overcloud) [stack@seal41 ~]$ openstack image save rhel-8.3 --file rhel-8.3-downloaded.qcow2 (overcloud) [stack@seal41 ~]$ sha512sum rhel-8.3-downloaded.qcow2 6acf19b94aaa6b4d9dfbddd2ec5ec07661f1ca4de8a3527e8ec03b20a51ca28e8faedcd22c78ef3c0bac4c0de8f6dc36356d0a589c699058421bff08c2cd4ac6 rhel-8.3-downloaded.qcow2 Again all the hash values are the same. Now one last time with Centos 8.3 (overcloud) [stack@seal41 ~]$ openstack image create --public --disk-format qcow2 --container-format bare --file CentOS-8-GenericCloud-8.3.2011-20201204.2.x86_64.qcow2 CentOS-8.3 +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | bf653cc2b5becb29c6cf7c6f7ecaf70f | | container_format | bare | | created_at | 2022-04-20T11:47:12Z | | disk_format | qcow2 | | file | /v2/images/924c9639-119a-45e0-92ef-bf10c2861495/file | | id | 924c9639-119a-45e0-92ef-bf10c2861495 | | min_disk | 0 | | min_ram | 0 | | name | CentOS-8.3 | | owner | 414eab17c9fa4d47be286609ee377514 | | properties | direct_url='cinder://51c8a48e-82a2-43ef-91fd-4c7869fb3416', os_hash_algo='sha512', os_hash_value='227f729dc5d5bd27058ac887652bf8c33b033b0a765ca9ec8f44da7bf9c1d89eeb3c7e5d0888f32dca9803220ac7259c9f4972236653d3c91b95414872090a9f', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 1275714048 | | status | active | | tags | | | updated_at | 2022-04-20T11:47:43Z | | virtual_size | None | | visibility | public | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@seal41 ~]$ sha512sum CentOS-8-GenericCloud-8.3.2011-20201204.2.x86_64.qcow2 227f729dc5d5bd27058ac887652bf8c33b033b0a765ca9ec8f44da7bf9c1d89eeb3c7e5d0888f32dca9803220ac7259c9f4972236653d3c91b95414872090a9f CentOS-8-GenericCloud-8.3.2011-20201204.2.x86_64.qcow2 (overcloud) [stack@seal41 ~]$ openstack image save CentOS-8.3 --file CentOS-8.3-downloaded.qcow2 (overcloud) [stack@seal41 ~]$ sha512sum CentOS-8.3-downloaded.qcow2 227f729dc5d5bd27058ac887652bf8c33b033b0a765ca9ec8f44da7bf9c1d89eeb3c7e5d0888f32dca9803220ac7259c9f4972236653d3c91b95414872090a9f CentOS-8.3-downloaded.qcow2 As can be seen above the sha512 value for all the source and uploaded images as well as the downloaded images/files are all identical per each image. Lets also test one of the images and boot an instance from a bootlabe volume created from one of the images: (overcloud) [stack@seal41 ~]$ nova boot --flavor large --block-device source=image,id=32fd077b-b54e-43ab-a667-87a2a101658a,dest=volume,size=20,shutdown=preserve,bootindex=0 RHEL8.3-instance --nic net-name=internal +--------------------------------------+-------------------------------------------------+ | Property | Value | +--------------------------------------+-------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | | | OS-EXT-SRV-ATTR:host | - | | OS-EXT-SRV-ATTR:hostname | rhel8-3-instance | | OS-EXT-SRV-ATTR:hypervisor_hostname | - | | OS-EXT-SRV-ATTR:instance_name | | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-t9e0603v | | OS-EXT-SRV-ATTR:root_device_name | - | | OS-EXT-SRV-ATTR:user_data | - | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | scheduling | | OS-EXT-STS:vm_state | building | | OS-SRV-USG:launched_at | - | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | adminPass | FSd5aTzBE5Mh | | config_drive | | | created | 2022-04-20T11:57:05Z | | description | - | | flavor:disk | 30 | | flavor:ephemeral | 0 | | flavor:extra_specs | {} | | flavor:original_name | large | | flavor:ram | 512 | | flavor:swap | 0 | | flavor:vcpus | 2 | | hostId | | | host_status | | | id | 5abb7e52-b2e6-487d-9855-6704cf6d5685 | | image | Attempt to boot from volume - no image supplied | | key_name | - | | locked | False | | locked_reason | - | | metadata | {} | | name | RHEL8.3-instance | | os-extended-volumes:volumes_attached | [] | | progress | 0 | | security_groups | default | | server_groups | [] | | status | BUILD | | tags | [] | | tenant_id | 414eab17c9fa4d47be286609ee377514 | | trusted_image_certificates | - | | updated | 2022-04-20T11:57:05Z | | user_id | 6c6a32bf562d48a598db99ba359d6765 | +--------------------------------------+-------------------------------------------------+ The instance is up/active (overcloud) [stack@seal41 ~]$ nova list +--------------------------------------+------------------+--------+------------+-------------+------------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------------+--------+------------+-------------+------------------------------------+ | 5abb7e52-b2e6-487d-9855-6704cf6d5685 | RHEL8.3-instance | ACTIVE | - | Running | internal=192.168.0.25 | We proved that image upload to glance of RHEL/centOS images works fine, good to verify.