Bug 888545 - glance image-create premature terminate with checksum leaks data on the glance image store
glance image-create premature terminate with checksum leaks data on the glanc...
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-glance (Show other bugs)
2.0 (Folsom)
Unspecified Unspecified
medium Severity medium
: snapshot3
: 2.1
Assigned To: Eoghan Glynn
Attila Fazekas
: Triaged
Depends On:
  Show dependency treegraph
Reported: 2012-12-18 16:05 EST by John Bresnahan
Modified: 2013-09-29 22:04 EDT (History)
2 users (show)

See Also:
Fixed In Version: openstack-glance-2012.2.3-1.el6ost
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-03-05 13:30:20 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1104924 None None None Never
Launchpad 1122299 None None None Never
OpenStack gerrit 20481 None None None Never

  None (edit)
Description John Bresnahan 2012-12-18 16:05:02 EST
Description of problem:

A data storage leak occurs in glance when a checksum is provided while uploading an image to glance and the client process uploading the image prematurely dies.  This results in a data on the file system, but no reference to that data in the registry.

Steps to Reproduce:
1. begin an upload of an image with the md5sum
2. terminate the client immediately
3. run 'glance index' and note that a new image has not appeared
4. run ls on the file system which is backing glance and notice that orphaned data is there

Actual results:

$ ls -l /gluster/glance/images/
total 0

$ glance index
ID                                   Name                           Disk Format          Container Format     Size          
------------------------------------ ------------------------------ -------------------- -------------------- --------------

$ glance  image-create  --name testit --container-format bare --disk-format qcow2 --is-public true --file f16-x86_64-openstack-sda.qcow2 --size 213581824 --checksum 755122332caeb9f661d5c978adb8b45f
^CTraceback (most recent call last):
  File "/usr/bin/glance", line 9, in <module>
    load_entry_point('python-glanceclient==0.5.1', 'console_scripts', 'glance')()
  File "/usr/lib/python2.6/site-packages/glanceclient/shell.py", line 432, in main
  File "/usr/lib/python2.6/site-packages/glanceclient/shell.py", line 403, in main
    args.func(client, args)
  File "/usr/lib/python2.6/site-packages/glanceclient/v1/shell.py", line 160, in do_image_create
    image = gc.images.create(**fields)
  File "/usr/lib/python2.6/site-packages/glanceclient/v1/images.py", line 212, in create
    'POST', '/v1/images', headers=hdrs, body=image_data)
  File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 192, in raw_request
    return self._http_request(url, method, **kwargs)
  File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 137, in _http_request
    conn.request(method, url, **kwargs)
  File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.6/httplib.py", line 954, in _send_request
  File "/usr/lib64/python2.6/httplib.py", line 756, in send
  File "<string>", line 1, in sendall

$ glance index
ID                                   Name                           Disk Format          Container Format     Size          
------------------------------------ ------------------------------ -------------------- -------------------- --------------

$ ls -l /gluster/glance/images/
total 103208
-rw-r-----. 1 glance glance 105684992 Dec 18 11:02 8d64fb62-67c5-4cbd-b109-123656d70903

Expected results:

The service should either report the image as existing but corrupt, giving the user a chance to clean it.  Or it should automatically deleted when it is detected as a failed upload.
Comment 2 Eoghan Glynn 2013-01-25 05:02:26 EST
Of the supported backend store types, this issue only impacts on the filesystem store.

(i.e. in the case of swift and s3, I've confirmed that partial uploads are properly cleaned up and do not leave behind partial image fragments).
Comment 3 Eoghan Glynn 2013-01-25 10:26:36 EST
Fix proposed to master upstream:

Comment 4 Eoghan Glynn 2013-01-25 10:27:06 EST
Fix proposed to stable/folsom upstream:

Comment 10 Eoghan Glynn 2013-02-11 11:46:40 EST
So it looks like the broken upload connection is not reported as an exception from read on RHEL (as was the case when I tested it in devstack on Ubuntu).

However the bug refers specifically to the case where the checksum (and size) is
explicitly specified for the new image. So even in the absence of an exceptional return from the read, we can still ensure deletion on a mismatched checksum or size.

Patch pending shortly ...
Comment 11 Eoghan Glynn 2013-02-11 13:56:31 EST
Further fix proposed upstream on master:

Comment 12 Eoghan Glynn 2013-02-11 13:56:58 EST
Further fix proposed upstream on stable/folsom:

Comment 13 Eoghan Glynn 2013-02-11 14:14:12 EST
Further fix proposed internally:

Comment 14 Eoghan Glynn 2013-02-11 15:45:46 EST
Merged internally.
Comment 16 Attila Fazekas 2013-02-20 13:26:53 EST
Working :)
Comment 18 errata-xmlrpc 2013-03-05 13:30:20 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.