Bug 1659264 - [RHOS 13] cinder ignores errors from glanceclient when creating volumes
Summary: [RHOS 13] cinder ignores errors from glanceclient when creating volumes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: 13.0 (Queens)
Assignee: Brian Rosmaita
QA Contact: Tzach Shefi
Kim Nylander
URL:
Whiteboard:
Depends On: 1664687
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-13 22:51 UTC by Eric Harney
Modified: 2019-03-14 13:47 UTC (History)
3 users (show)

Fixed In Version: openstack-cinder-12.0.4-7.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, a code change in the Pike release introduced a regression, causing Cinder to ignore some IOError exceptions that the glanceclient raised when downloading an image. If one of these ignored exceptions occurred, a volume could be created with truncated or corrupt data. With this update, the code was modified and these exceptions are no longer ignored. As a result, when an IOError occurs during image download, Cinder logs and handles the exception correctly.
Clone Of:
: 1664687 (view as bug list)
Environment:
Last Closed: 2019-03-14 13:47:53 UTC
Target Upstream Version:

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1799221 0 None None None 2018-12-13 22:51:57 UTC
OpenStack gerrit 629463 0 'None' 'MERGED' 'cinder-volume: Stop masking IOError different than ENOSPC' 2019-11-26 09:08:04 UTC
Red Hat Product Errata RHBA-2019:0560 0 None None None 2019-03-14 13:47:55 UTC

Description Eric Harney 2018-12-13 22:51:58 UTC 
Cinder ignores some glanceclient errors (such as failed checksum validation) when creating volumes from images.

https://bugs.launchpad.net/cinder/+bug/1799221
Comment 2 Alan Bishop 2018-12-19 15:04:58 UTC 
Upstream patch has been submitted by someone else, so we need to track its progress and handle the backports.
Comment 4 Brian Rosmaita 2019-01-10 04:56:06 UTC 
This isn't going to be a simple backport from Rocky to Queens upstream.  See https://bugs.launchpad.net/cinder/+bug/1799221/comments/12 for more info.
Comment 13 Tzach Shefi 2019-02-26 11:45:25 UTC 
Tip to self for verification steps follow:
https://bugzilla.redhat.com/show_bug.cgi?id=1664687#c10
Comment 14 Brian Rosmaita 2019-02-26 13:45:39 UTC 
Minor note: there's no 'multihash' (os_hash_algo + os_hash_value) in Queens Glance, so you'll have to just modify the 'checksum' field.
Comment 17 Tzach Shefi 2019-02-28 13:35:25 UTC 
Verified on: 
openstack-cinder-12.0.4-8.el7ost.noarch

Thanks Brian per your tip :) 

Uploaded image to glance:
#glance image-create --disk-format qcow2 --container-format bare --name cirros.bad --file cirros-0.3.5-i386-disk.img

I then changed checksum via SQL access

Before change:
| a43dba56-4732-4ca0-9050-c849cb6252ed | cirros.bad | 40602 | active | 2019-02-28 13:19:52 | 2019-02-28 13:19:53 | NULL       |       0 | qcow2       | bare             | 0e4128f7bacfd887b8dbf450b46357bc | e7979f642cfa40e8872bdd11ce912505 |        0 |       0 |         0 |         NULL | shared     |


MariaDB [glance]> update images set checksum="0e4128f7bacfd887b8dbf450b4635zzz"  where name="cirros.bad";

| a43dba56-4732-4ca0-9050-c849cb6252ed | cirros.bad | 40602 | active | 2019-02-28 13:19:52 | 2019-02-28 13:19:53 | NULL       |       0 | qcow2       | bare             | 0e4128f7bacfd887b8dbf450b4635zzz | e7979f642cfa40e8872bdd11ce912505 |        0 |       0 |         0 |         NULL | shared     |


Now try to create a volume to said image, should fail

#cinder create 1 --image cirros.bad ->
| id                             | c034d180-b04a-4927-8b89-5b605603e41f  

Cinder list -> 
 c034d180-b04a-4927-8b89-5b605603e41f | error     | -            | 1    | -           | false    |                                      |


Cinder create fails as expected.
logs reports the expected error, so were good to verify -> 

/var/log/containers/cinder/cinder-volume.log:1313:2019-02-28 13:27:14.599 70 ERROR oslo_messaging.rpc.server ImageDownloadFailed: Failed to download image a43dba56-4732-4ca0-9050-c849cb6252ed, reason: IOError: 32 Corrupt image download. Checksum was 0e4128f7bacfd887b8dbf450b46357bc expected 0e4128f7bacfd887b8dbf450b4635zzz
Comment 19 errata-xmlrpc 2019-03-14 13:47:53 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0560
Note You need to log in before you can comment on or make changes to this bug.