Bug 912744 - Live migration fails. Libvirt says Cannot assign requested address.
Summary: Live migration fails. Libvirt says Cannot assign requested address.
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 2.0 (Folsom)
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
: 2.1
Assignee: Daniel Berrangé
QA Contact: Jaroslav Henner
URL:
Whiteboard:
Depends On:
Blocks: 958057 1180600 1180602
TreeView+ depends on / blocked
 
Reported: 2013-02-19 15:04 UTC by Jaroslav Henner
Modified: 2023-09-18 09:58 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Release Note
Doc Text:
If the vncserver_listen parameter in /etc/nova/nova.conf is set to the IP address of host, then it will no longer be possible to migrate virtual machine instances to another host. Attempts at migration will result in an error when starting the guest on the target host. To workaround this flaw, the listen address must be set to the wildcard address (e.g. 0.0.0.0). Firewall rules on the host must then be used to lock down access to the VNC console port number range (as configured in /etc/libvirt/qemu.conf, to only those networks which should have access. In particular care must be taken to prevent virtual machines instances from accessing the VNC ports on the host, if the host and guest share any networks.
Clone Of:
: 958057 1178924 (view as bug list)
Environment:
Last Closed: 2013-04-30 09:57:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Jaroslav Henner 2013-02-19 15:04:15 UTC
Description of problem:
Having one controller, two nodes, VM migration fails because libvirt is trying to bind to address of the  source host.

Version-Release number of selected component (if applicable):
openstack-nova-compute-2012.2.3-1.el6ost.noarch
openstack-packstack-2012.2.2-1.0.dev408.el6ost.noarch
openstack-nova-2012.2.3-1.el6ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. Deploy with packstack
2. mount /var/lib/nova/instances from some NFS
3. nova boot ...
4. nova live-migration ...
  
Actual results:
2013-02-19 14:55:52 DEBUG nova.network.linux_net [req-28b68fc1-7a96-41a0-bb69-f0c376587bab 4b5c555a29ca4321b448ea5dade6ef03 3d62aba4b5904c5fb4c17b70aa5a11cd] IPTablesManager.apply completed with success _apply /usr/lib/python2.6/site-packages/nova/network/linux_net.py:374
2013-02-19 14:55:53 4945 ERROR nova.virt.libvirt.driver [-] [instance: 78e5a9b8-526f-4ec8-a8cf-54a60ad44dcb] Live Migration failure: internal error Process exited while reading console log output: char device redirected to /dev/pts/0
inet_listen_opts: bind(ipv4,10.34.69.2,5900): Cannot assign requested address
inet_listen_opts: FAILED
Failed to start VNC server on `10.34.69.2:0'



Expected results:
migration completed

Additional info:

Comment 2 Jaroslav Henner 2013-02-19 15:06:17 UTC
Created attachment 699494 [details]
logs.tar.xz

Comment 3 Daniel Berrangé 2013-02-19 15:08:14 UTC
Can you provide the /etc/nova/nova.conf file from *both* hosts. Also provide the /var/log/libvirt/qemu/instance-XXXXXXX.log file from *both* hosts involved.

Comment 6 Daniel Berrangé 2013-02-19 15:26:53 UTC
Ok, so the problem is that on the first host there is vncserver_listen=10.34.69.2 in nova.conf. This gets into the guest XML as  listen="10.33.69.2" which is fine on the source host. During migration the XML from the source is sent across to the destination host, where this IP address is just wrong.

If Nova is going to use host-specific IP addresses in the XML, then it needs to override the XML used during migration to provide an updated IP address.

A simpler option is to not hard code the IP address at all - use 0.0.0.0 to make it listen on all interfaces.


Note You need to log in before you can comment on or make changes to this bug.