Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 912744

Summary: Live migration fails. Libvirt says Cannot assign requested address.
Product: Red Hat OpenStack Reporter: Jaroslav Henner <jhenner>
Component: openstack-novaAssignee: Daniel Berrangé <berrange>
Status: CLOSED DEFERRED QA Contact: Jaroslav Henner <jhenner>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.0 (Folsom)CC: breeler, derekh, ndipanov, sgordon
Target Milestone: ---   
Target Release: 2.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Release Note
Doc Text:
If the vncserver_listen parameter in /etc/nova/nova.conf is set to the IP address of host, then it will no longer be possible to migrate virtual machine instances to another host. Attempts at migration will result in an error when starting the guest on the target host. To workaround this flaw, the listen address must be set to the wildcard address (e.g. 0.0.0.0). Firewall rules on the host must then be used to lock down access to the VNC console port number range (as configured in /etc/libvirt/qemu.conf, to only those networks which should have access. In particular care must be taken to prevent virtual machines instances from accessing the VNC ports on the host, if the host and guest share any networks.
 Story Points: ---
Clone Of:
: 958057 1178924 (view as bug list) Environment:
Last Closed: 2013-04-30 09:57:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 958057, 1180600, 1180602    

Description Jaroslav Henner 2013-02-19 15:04:15 UTC 
Description of problem:
Having one controller, two nodes, VM migration fails because libvirt is trying to bind to address of the  source host.

Version-Release number of selected component (if applicable):
openstack-nova-compute-2012.2.3-1.el6ost.noarch
openstack-packstack-2012.2.2-1.0.dev408.el6ost.noarch
openstack-nova-2012.2.3-1.el6ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. Deploy with packstack
2. mount /var/lib/nova/instances from some NFS
3. nova boot ...
4. nova live-migration ...
  
Actual results:
2013-02-19 14:55:52 DEBUG nova.network.linux_net [req-28b68fc1-7a96-41a0-bb69-f0c376587bab 4b5c555a29ca4321b448ea5dade6ef03 3d62aba4b5904c5fb4c17b70aa5a11cd] IPTablesManager.apply completed with success _apply /usr/lib/python2.6/site-packages/nova/network/linux_net.py:374
2013-02-19 14:55:53 4945 ERROR nova.virt.libvirt.driver [-] [instance: 78e5a9b8-526f-4ec8-a8cf-54a60ad44dcb] Live Migration failure: internal error Process exited while reading console log output: char device redirected to /dev/pts/0
inet_listen_opts: bind(ipv4,10.34.69.2,5900): Cannot assign requested address
inet_listen_opts: FAILED
Failed to start VNC server on `10.34.69.2:0'



Expected results:
migration completed

Additional info:
Comment 2 Jaroslav Henner 2013-02-19 15:06:17 UTC 
Created attachment 699494 [details]
logs.tar.xz
Comment 3 Daniel Berrangé 2013-02-19 15:08:14 UTC 
Can you provide the /etc/nova/nova.conf file from *both* hosts. Also provide the /var/log/libvirt/qemu/instance-XXXXXXX.log file from *both* hosts involved.
Comment 6 Daniel Berrangé 2013-02-19 15:26:53 UTC 
Ok, so the problem is that on the first host there is vncserver_listen=10.34.69.2 in nova.conf. This gets into the guest XML as  listen="10.33.69.2" which is fine on the source host. During migration the XML from the source is sent across to the destination host, where this IP address is just wrong.

If Nova is going to use host-specific IP addresses in the XML, then it needs to override the XML used during migration to provide an updated IP address.

A simpler option is to not hard code the IP address at all - use 0.0.0.0 to make it listen on all interfaces.