Description of problem: Anomalous behavior during verification for BZ #1664687 made me suspect that openstack-cinder 14z1 was packaged with an outdated python-glanceclient. How reproducible: When a Glance image has an incorrect os_hash_value, an exception should occur during image download and Cinder should refuse to create a volume from the image. If the legacy Glance 'checksum' (md5 hash) value is *also* incorrect, however, Cinder will correctly refuse to create the volume. Steps to Reproduce: 1. Upload an image to Glance, verify that you can create a cinder volume from it. 2. Go to the Glance database and modify the 'os_hash_value' for that image (e.g., replace the first or last characters with 'zzz') 3. Create a cinder volume from the image. Actual results: A volume is created from the image. Expected results: Cinder reports that it cannot create a volume from the specified image. The cinder-volume log should contain an ERROR message stating "Corrupt image download. Hash was c756d...b508e expected c756d...b5zzz"
Glanceclient 2.12.1 was originally released as the "rocky-final" version. It did not contain multihash download validation. A subsequent stable-rocky-bugfix version, glanceclient 2.13.0, was released on 2018-10-31. The current stable rocky glanceclient is 2.13.1. See https://launchpad.net/python-glanceclient/rocky for more info about the Rocky glanceclient releases.
It looks like after the last update we ship python2-glanceclient-2.13.1-0.20181219194754.e0673a1.el7ost.noarch.rpm . Does it mean that this bug can be closed?
The problem was the version of glanceclient that wound up in the cinder container. The RDO patch referenced on this BZ changed the openstack-cinder.spec to make sure the requirements specified glanceclient>=2.13.1. (It had been >=2.8.0, which I guess is how we got 2.12.1 in the cinder container I mentioned in comment #3). That change to the spec file made it to the downstream distgit with this commit: commit a31973651f0c4ab7129b1b23dd09a95425a7d7d1 Author: Steve Linabery <slinaber> Date: Mon Mar 25 21:47:21 2019 -0500 Import of openstack-cinder from RDO Time: 2019-03-25 20:30:07 NVR: openstack-cinder-13.0.4-0.20190311234911.96329a6.el7 Source Repo: git://git.openstack.org/openstack/cinder Source SHA: 96329a6337e0b602d35c67d9a67e1036c0e16b86 Dist Repo: https://github.com/rdo-packages/cinder-distgit.git Dist SHA: 56ac10e69609c11e733d298b05a955395f419c31 I looked at the 14z2 puddle http://download.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/14.0-RHEL-7/z2/, but I don't know how to verify that cinder will be using the correct glanceclient.
Verified on: Wasn't sure which fixed-in I should compare with, as field is blank. Any way these below are taken from within c-vol docker. python-cinder-13.0.5-2.el7ost.noarch python2-cinderclient-4.0.1-0.20180809133302.460229c.el7ost.noarch puppet-cinder-13.3.3-0.20190420065552.c5967d5.el7ost.noarch openstack-cinder-13.0.5-2.el7ost.noarch python2-glanceclient-2.13.1-0.20181219194754.e0673a1.el7ost.noarch puppet-glance-13.3.2-0.20190420084238.45584f0.el7ost.noarch Uploaded cirros image (overcloud) [stack@undercloud-0 ~]$ glance image-list +--------------------------------------+--------+ | ID | Name | +--------------------------------------+--------+ | 4bc7fd10-5ece-4f91-b381-f196c4f021c2 | cirros | Created a volume from said image (overcloud) [stack@undercloud-0 ~]$ cinder create 1 --image cirros +--------------------------------+---------------------------------------+ | Property | Value | +--------------------------------+---------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2019-06-02T11:58:38.000000 | | description | None | | encrypted | False | | id | 8c4b0485-6e17-4d50-88c1-90f2a213486e | | metadata | {} | | migration_status | None | | multiattach | False | | name | None | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 541737fd5c774090bc30dab74f7e41a3 | | replication_status | None | | size | 1 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | 2019-06-02T11:58:38.000000 | | user_id | 6026360b4d0541bea7ab13e8f16131d2 | | volume_type | tripleo | +--------------------------------+---------------------------------------+ Available -> | 8c4b0485-6e17-4d50-88c1-90f2a213486e | available | - | 1 | tripleo | true | | Show image before disturbing hash (overcloud) [stack@undercloud-0 ~]$ glance image-show 4bc7fd10-5ece-4f91-b381-f196c4f021c2 +----------------------------------+----------------------------------------------------------------------------------+ | Property | Value | +----------------------------------+----------------------------------------------------------------------------------+ | checksum | 7316af7358dd32ca1956d72ac2c9e147 | | container_format | bare | | created_at | 2019-06-02T11:43:13Z | | direct_url | swift+config://ref1/glance/4bc7fd10-5ece-4f91-b381-f196c4f021c2 | | disk_format | qcow2 | | id | 4bc7fd10-5ece-4f91-b381-f196c4f021c2 | | min_disk | 0 | | min_ram | 0 | | name | cirros | | os_hash_algo | sha512 | | os_hash_value | 734c9281adf72b9947eb9ab85f5e9db0fe388b742ebb68469d4e87d17065e7a39b501c184e60913c | | | 79ec9e79042992024a2063b906c66a5c37735626f8f14bae | | os_hidden | False | | owner | 541737fd5c774090bc30dab74f7e41a3 | | owner_specified.openstack.md5 | 7316af7358dd32ca1956d72ac2c9e147 | | owner_specified.openstack.object | images/cirros | | owner_specified.openstack.sha256 | 53ef7941846012d031db80d2936b23de65c912ae45a1a68b21adb661f080a21b | | protected | False | | size | 12528640 | | status | active | | tags | [] | | updated_at | 2019-06-02T11:43:15Z | | virtual_size | Not available | | visibility | public | +----------------------------------+----------------------------------------------------------------------------------+ inside db docker MariaDB [glance]> UPDATE images SET os_hash_value="734c9281adf72b9947eb9ab85f5e9db0fe388b742ebb68469d4e87d17065e7a39b501c184e60913c79ec9e79042992024a2063b906c66a5c37735626f8f14zzz" WHERE id = "4bc7fd10-5ece-4f91-b381-f196c4f021c2"; Then we see hash has changed notice zzz at the end. (overcloud) [stack@undercloud-0 ~]$ glance image-show 4bc7fd10-5ece-4f91-b381-f196c4f021c2 +----------------------------------+----------------------------------------------------------------------------------+ | Property | Value | +----------------------------------+----------------------------------------------------------------------------------+ | checksum | 7316af7358dd32ca1956d72ac2c9e147 | | container_format | bare | | created_at | 2019-06-02T11:43:13Z | | direct_url | swift+config://ref1/glance/4bc7fd10-5ece-4f91-b381-f196c4f021c2 | | disk_format | qcow2 | | id | 4bc7fd10-5ece-4f91-b381-f196c4f021c2 | | min_disk | 0 | | min_ram | 0 | | name | cirros | | os_hash_algo | sha512 | | os_hash_value | 734c9281adf72b9947eb9ab85f5e9db0fe388b742ebb68469d4e87d17065e7a39b501c184e60913c | | | 79ec9e79042992024a2063b906c66a5c37735626f8f14zzz Now we try to create a volume from image again, as expected image create from said volume now fails (overcloud) [stack@undercloud-0 ~]$ cinder list +--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+ | 00cc00b4-6b4b-4788-a90f-7ff53c9fa32a | error | postzzz | 1 | tripleo | false | | | 8c4b0485-6e17-4d50-88c1-90f2a213486e | available | - | 1 | tripleo | true | | +--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+ c-vol log error 2903:2019-06-02 12:17:30.450 69 ERROR oslo_messaging.rpc.server ImageDownloadFailed: Failed to download image 4bc7fd10-5ece-4f91-b381-f196c4f021c2, reason: IOError: 32 Corrupt image download. Hash was 734c9281adf72b9947eb9ab85f5e9db0fe388b742ebb68469d4e87d17065e7a39b501c184e60913c79ec9e79042992024a2063b906c66a5c37735626f8f14bae expected 734c9281adf72b9947eb9ab85f5e9db0fe388b742ebb68469d4e87d17065e7a39b501c184e60913c79ec9e79042992024a2063b906c66a5c37735626f8f14zzz Looking good to verify.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1678