A. Description of problem: Cinder volumes created from images are marked as available even though they were not successfully transferred and were unbootable. B. Version-Release number of selected component (if applicable): openstack-cinder-2014.1.3-2.el6ost.noarch openstack-glance-2014.1.3-1.el6ost.noarch python-cinder-2014.1.3-2.el6ost.noarch python-cinderclient-1.0.9-1.el6ost.noarch python-glance-2014.1.3-1.el6ost.noarch python-glanceclient-0.12.0-2.el6ost.noarch C. How reproducible: Only occurs during a load test 25% of the time. D Steps to Reproduce: 1. Boot 60 instances from the same qcow2 image 2. Observe that approximately 15 will not boot 3. Inspect each of the 15 and see that their cinder volume is not an accurate bootable copy of the image E. Actual results: - 100% of volumes report that they are available - 25% of volumes are unbootable and should not have been marked as available F. Expected results: If volumes were unable to boot, then they should not have been marked as available. E. Additional info: Using 3par cinder driver https://blueprints.launchpad.net/cinder/+spec/hp3par-volume-driver
From the logs, it seems that this is not a Glance issue. The image is downloaded correctly and the checksum verification doesn't fail. Furthermore, the image is successfully copied into the volume. Then the image metadata is also copied to the volume metadata. Would it me possible to have compute's logs as well? According to the logs, the bootable flag is being set to the volume as well.
Yes, I will get the logs from the compute nodes.
An additional test was conducted. - Create 128 VMs, 4 at once, then delete them and do this 32 times - Of the 128 VMs created, 34 of them had the symptoms of this bug
An additional test was conducted after setting volume_clear=zero in cinder.conf. - Create 512 VMs, 8 at once, then delete them and do this 64 times - Of 512 VMs created, 444 passed the test but 68 had the symptoms of this bug
We are working to provide an official hotfix. I will update this bug with the list of the relevant fixes once we complete review process.
Fixed here https://bugzilla.redhat.com/show_bug.cgi?id=1255523#c20
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2686.html