Description of problem: A data storage leak occurs in glance when a checksum is provided while uploading an image to glance and the client process uploading the image prematurely dies. This results in a data on the file system, but no reference to that data in the registry. Steps to Reproduce: 1. begin an upload of an image with the md5sum 2. terminate the client immediately 3. run 'glance index' and note that a new image has not appeared 4. run ls on the file system which is backing glance and notice that orphaned data is there Actual results: $ ls -l /gluster/glance/images/ total 0 $ glance index ID Name Disk Format Container Format Size ------------------------------------ ------------------------------ -------------------- -------------------- -------------- $ glance image-create --name testit --container-format bare --disk-format qcow2 --is-public true --file f16-x86_64-openstack-sda.qcow2 --size 213581824 --checksum 755122332caeb9f661d5c978adb8b45f ^CTraceback (most recent call last): File "/usr/bin/glance", line 9, in <module> load_entry_point('python-glanceclient==0.5.1', 'console_scripts', 'glance')() File "/usr/lib/python2.6/site-packages/glanceclient/shell.py", line 432, in main OpenStackImagesShell().main(sys.argv[1:]) File "/usr/lib/python2.6/site-packages/glanceclient/shell.py", line 403, in main args.func(client, args) File "/usr/lib/python2.6/site-packages/glanceclient/v1/shell.py", line 160, in do_image_create image = gc.images.create(**fields) File "/usr/lib/python2.6/site-packages/glanceclient/v1/images.py", line 212, in create 'POST', '/v1/images', headers=hdrs, body=image_data) File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 192, in raw_request return self._http_request(url, method, **kwargs) File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 137, in _http_request conn.request(method, url, **kwargs) File "/usr/lib64/python2.6/httplib.py", line 914, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.6/httplib.py", line 954, in _send_request self.send(body) File "/usr/lib64/python2.6/httplib.py", line 756, in send self.sock.sendall(data) File "<string>", line 1, in sendall KeyboardInterrupt $ glance index ID Name Disk Format Container Format Size ------------------------------------ ------------------------------ -------------------- -------------------- -------------- $ ls -l /gluster/glance/images/ total 103208 -rw-r-----. 1 glance glance 105684992 Dec 18 11:02 8d64fb62-67c5-4cbd-b109-123656d70903 Expected results: The service should either report the image as existing but corrupt, giving the user a chance to clean it. Or it should automatically deleted when it is detected as a failed upload.
Of the supported backend store types, this issue only impacts on the filesystem store. (i.e. in the case of swift and s3, I've confirmed that partial uploads are properly cleaned up and do not leave behind partial image fragments).
Fix proposed to master upstream: https://review.openstack.org/20472
Fix proposed to stable/folsom upstream: https://review.openstack.org/20481
So it looks like the broken upload connection is not reported as an exception from read on RHEL (as was the case when I tested it in devstack on Ubuntu). However the bug refers specifically to the case where the checksum (and size) is explicitly specified for the new image. So even in the absence of an exceptional return from the read, we can still ensure deletion on a mismatched checksum or size. Patch pending shortly ...
Further fix proposed upstream on master: https://review.openstack.org/21675
Further fix proposed upstream on stable/folsom: https://review.openstack.org/21678
Further fix proposed internally: https://code.engineering.redhat.com/gerrit/3103
Merged internally.
Working :)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0593.html