Bug 1085005 - openstack-nova: several instances are able to be configure the same bootable volume
Summary: openstack-nova: several instances are able to be configure the same bootable ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 4.0
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: z4
: 4.0
Assignee: Nikola Dipanov
QA Contact: Yogev Rabl
URL:
Whiteboard: storage
Depends On:
Blocks: 1020501
TreeView+ depends on / blocked
 
Reported: 2014-04-07 13:59 UTC by Vladan Popovic
Modified: 2022-07-09 06:17 UTC (History)
12 users (show)

Fixed In Version: openstack-nova-2013.2.1-1.el6ost
Doc Type: Bug Fix
Doc Text:
Cause: There is a race condition with checking the status of requested volumes in the API, and combined with the fact that we would attempt to reschedule a failed instance to a different host, they can cause a volume to be "stolen" from an instance that managed to attach it successfully. Consequence: If several instances get requested close to each other, and they request the same volume, it was possible for an instance that got rescheduled due to failing the volume setup (for the already taken volume), to disconnect that volume from the instance that already had it attached in the reschedule process. Fix: If an instance fails the volume setup during boot (due to the volume being set up by a different instance), we will not attempt to re-schedule the failed instance to a different host and thus avoid disconnecting a completely attached volume. Result: Only one instance that requested the same volume succeeds while all others go into the ERROR state.
Clone Of: 1020501
Environment:
Last Closed: 2014-05-29 20:35:40 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1195947 0 None None None Never
OpenStack gerrit 54916 0 None None None Never
Red Hat Product Errata RHSA-2014:0578 0 normal SHIPPED_LIVE Moderate: openstack-nova security, bug fix, and enhancement update 2014-05-30 00:27:05 UTC

Comment 1 Dafna Ron 2014-04-07 14:12:16 UTC 
We will verify this bug so I am taking it. 
to verify: 
1. run the original script booting instances from the volume 
2. make sure that only one instance move to available while the rest move to ERROR. 
3. make sure the volume is not damaged (detach/reattach the volume)
4. run the same script but instead of booting instances from the volume - attach the volume to several running instances.
Comment 6 Nikola Dipanov 2014-04-10 11:47:56 UTC 
Dafna - Comment 1 sounds like a good plan to test it.

Attaching volumes should not really be part of this bug, as the issue was with how we handled rescheduling on failures. But I do urge you to test attaching as well and possibly report a different bug.
Comment 11 Nikola Dipanov 2014-04-30 08:44:23 UTC 
Moving back to 4.0 as this is indeed fixed in 4.0
Comment 14 Yogev Rabl 2014-05-04 13:24:36 UTC 
verified on:

python-novaclient-2.15.0-4.el6ost.noarch
openstack-nova-conductor-2013.2.3-7.el6ost.noarch
openstack-nova-scheduler-2013.2.3-7.el6ost.noarch
openstack-nova-common-2013.2.3-7.el6ost.noarch
openstack-nova-api-2013.2.3-7.el6ost.noarch
openstack-nova-console-2013.2.3-7.el6ost.noarch
openstack-nova-network-2013.2.3-7.el6ost.noarch
openstack-nova-cert-2013.2.3-7.el6ost.noarch
python-nova-2013.2.3-7.el6ost.noarch
openstack-nova-compute-2013.2.3-7.el6ost.noarch
openstack-nova-novncproxy-2013.2.3-7.el6ost.noarch
python-cinderclient-1.0.7-2.el6ost.noarch
Comment 16 errata-xmlrpc 2014-05-29 20:35:40 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2014-0578.html
Note You need to log in before you can comment on or make changes to this bug.