Bug 1196364 - Live Migration doesn't handle RBD Storage failures
Summary: Live Migration doesn't handle RBD Storage failures
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 6.0 (Juno)
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: z3
: 6.0 (Juno)
Assignee: Pádraig Brady
QA Contact: Yogev Rabl
URL:
Whiteboard:
Depends On:
Blocks: rhelosp_ceph_integration
TreeView+ depends on / blocked
 
Reported: 2015-02-25 19:55 UTC by Pádraig Brady
Modified: 2023-02-22 23:02 UTC (History)
13 users (show)

Fixed In Version: openstack-nova-2014.2.3-3.el7ost
Doc Type: Bug Fix
Doc Text:
The libvirt driver previously handled shared file systems incorrectly (for example, using NFS or Ceph), and live migration would fail or hang with exceptions. With this update, raw or qcow images are now treated appropriately on shared file systems, and live migration now works reliably and efficiently.
Clone Of: 1196350
Environment:
Last Closed: 2015-05-05 13:30:16 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 134693 0 None None None Never
Red Hat Product Errata RHBA-2015:0931 0 normal SHIPPED_LIVE openstack-nova bug fix advisory 2015-05-05 17:29:39 UTC

Comment 2 Sean Toner 2015-04-20 15:23:44 UTC 
I'm not sure what needs to be done to reproduce and test the fix.

For example, is it sufficient to create an NFS setup, perform some live migrations to verify that the base scenario works, and then disable (or maybe some of the dependency services, like rpcbind) and see how nova handles a live migration?

I'm assuming that if NFS (or ceph or gluster) fails, nova will fail gracefully?
Comment 3 Sean Toner 2015-04-23 16:26:23 UTC 
From the doc text, it appears that either NFS or ceph can be used for verification.  If this is true, is NFS as shared storage sufficient to test this code path?

I can set up live migration wit NFS pretty easily, but ceph will require me to work with yogev
Comment 4 Pádraig Brady 2015-04-23 16:28:42 UTC 
nfs is fine
Comment 5 Yogev Rabl 2015-04-30 11:42:44 UTC 
verified with Ceph as the back end of Nova. 
version:
openstack-nova-common-2014.2.3-9.el7ost.noarch
openstack-nova-cert-2014.2.3-9.el7ost.noarch
openstack-nova-compute-2014.2.3-9.el7ost.noarch
python-novaclient-2.20.0-1.el7ost.noarch
python-nova-2014.2.3-9.el7ost.noarch
openstack-nova-novncproxy-2014.2.3-9.el7ost.noarch
openstack-nova-console-2014.2.3-9.el7ost.noarch
openstack-nova-scheduler-2014.2.3-9.el7ost.noarch
openstack-nova-conductor-2014.2.3-9.el7ost.noarch
openstack-nova-api-2014.2.3-9.el7ost.noarch

Setup topology:
host01 - controller + compute node
host02 - compute node

from /etc/nova/nova.conf (on both computes):

[libvirt]
virt_type=kvm
inject_password=false
inject_key=false
inject_partition = -2
rbd_user = <ceph user>
rbd_secret_uuid = <uuid>
images_type = rbd
images_rbd_pool = <pool name>
images_rbd_ceph_conf = /etc/ceph/ceph.conf
live_migration_flag = VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE

Steps:
1. launch an instance 
2. ssh to the instance 
3. live migrate the instance 
4. verify the migration in the second host
Comment 7 errata-xmlrpc 2015-05-05 13:30:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0931.html
Note You need to log in before you can comment on or make changes to this bug.