Bug 971485

Summary: Live migration fails if "vnc_listen" set in /etc/libvirt/qemu.conf
Product: Red Hat Enterprise Linux 6 Reporter: Roman <rommer>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.4CC: acathrow, dallan, dyuan, lcui, pasquale.demichele, ydu, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-20.el6 Doc Type: Bug Fix
Doc Text:
Cause: If user specified no vnc listen address in their domain XML, the one from qemu.conf is used. However, uppon migrating, there was no difference between cases where the listen address was set by user in XML directly or copied from qemu.conf. Consequence: A domain couldn't be migrated. Fix: If the listen address is copied from qemu.conf, it is not transferred to the destination (which probably can't bind() to the old listen address anyway). Result: A domain can be migrated without any error.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-21 09:02:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Roman 2013-06-06 16:01:38 UTC
Description of problem:
Live migration fails if "vnc_listen" set in /etc/libvirt/qemu.conf

Version-Release number of selected component (if applicable):
libvirt-0.10.2-18.el6_4.5.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Setup vnc_listen option /etc/libvirt/qemu.conf to any ip.
2. Restart libvirtd, start any VM
3. Start live migration:
# virsh migrate --timeout 300 --live --verbose <VM> qemu+tcp://other-host/system

Actual results:
error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/1
inet_listen_opts: bind(ipv4,<IP>,5900): Cannot assign requested address
inet_listen_opts: FAILED
Failed to start VNC server on `<IP>:0'

Expected results:
migration completed

Additional info:

Comment 2 Roman 2013-06-07 16:14:23 UTC
Patch that breaks live migration:
http://libvirt.org/git/?p=libvirt.git;a=commit;h=d7f9d827531bc843b7c5aa9d3e8c08738a1de248

Comment 3 Dave Allan 2013-06-07 16:22:10 UTC
Roman, thanks for the bisect; that makes sense.  Michal, can you see what can be done about this?

Comment 4 Roman 2013-06-08 21:56:23 UTC
Simple workaround using libvirt hooks:

# cat /etc/libvirt/hooks/qemu-filter.xsl 
<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output omit-xml-declaration="yes"/>
  <xsl:template match="/domain/devices/graphics/listen"/>
  <xsl:template match="/domain/devices/graphics/@listen"/>
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

# cat /etc/libvirt/hooks/qemu
#!/bin/bash

[ "$2" = "migrate" ] && xsltproc /etc/libvirt/hooks/qemu-filter.xsl -

exit 0

Comment 5 Michal Privoznik 2013-06-10 17:39:00 UTC
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2013-June/msg00456.html

Comment 6 Michal Privoznik 2013-06-11 12:58:20 UTC
I've just pushed the patch upstream:

commit 6546017c50c104d0b9867137b64ab1f4a312e436
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Jun 10 17:51:46 2013 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jun 11 14:11:46 2013 +0200

    qemu_migrate: Dispose listen address if set from config
    
    https://bugzilla.redhat.com/show_bug.cgi?id=971485
    
    As of d7f9d827531bc843b7c5aa9d3e8c08738a1de248 we copy the listen
    address from the qemu.conf config file in case none has been provided
    via XML. But later, when migrating, we should not include such listen
    address in the migratable XML as it is something autogenerated, not
    requested by user. Moreover, the binding to the listen address will
    likely fail, unless the address is '0.0.0.0' or its IPv6 equivalent.
    This patch introduces a new boolean attribute to virDomainGraphicsListenDef
    to distinguish autofilled listen addresses. However, we must keep the
    attribute over libvirtd restarts, so it must be kept within status XML.

v1.0.6-61-g6546017

Comment 9 yanbing du 2013-07-16 08:23:47 UTC
# rpm -q libvirt
libvirt-0.10.2-20.el6.x86_64

Following the steps in comment0, when start migration, libvirtd on dest host will crash, which should cause by bug 984793.

Comment 10 yanbing du 2013-07-23 06:29:24 UTC
1. Set vnc_listen="10.66.106.30" in /etc/libvirt/qemu.conf
2. Start a guest
# virsh dumpxml spice
...
    <graphics type='vnc' port='5900' autoport='yes' listen='10.66.106.30'>
      <listen type='address' address='10.66.106.30'/>
    </graphics>
...

3. Do live migration
3.1 With libvirt-0.10.2-18.el6_4.5, when do migration:
# virsh migrate --timeout 300 --live --verbose spice qemu+tcp://10.66.106.31/system
error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/2
inet_listen_opts: bind(ipv4,10.66.106.30,5900): Cannot assign requested address
inet_listen_opts: FAILED
Failed to start VNC server on `10.66.106.30:0'

3.2. With libvirt-0.10.2-21.el6, when do migration:
# virsh migrate --timeout 300 --live --verbose spice qemu+tcp://10.66.106.31/system
Migration: [100 %]

and on the dest host:
# virsh list
 Id    Name                           State
----------------------------------------------------
 1     spice                          running

as vnc_listen="10.66.106.31 on the dest host qemu.conf, the guest listen address also changed
# virsh dumpxml spice
...
    <graphics type='vnc' port='5900' autoport='yes' listen='10.66.106.31'>
      <listen type='address' address='10.66.106.31'/>
    </graphics>
...

So move this bug to VERIFIED.

Comment 12 errata-xmlrpc 2013-11-21 09:02:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1581.html

Comment 13 pasquale.demichele 2014-06-14 11:26:33 UTC
(In reply to errata-xmlrpc from comment #12)
> Since the problem described in this bug report should be
> resolved in a recent advisory, it has been closed with a
> resolution of ERRATA.
> 
> For information on the advisory, and where to find the updated
> files, follow the link below.
> 
> If the solution does not work for you, open a new bug report.
> 
> http://rhn.redhat.com/errata/RHBA-2013-1581.html

Hi all.

I have the same issue with this packages:

$ rpm -qa | grep libvirt

libvirt-python-0.10.2-29.el6_5.3.x86_64
libvirt-client-0.10.2-29.el6_5.3.x86_64
libvirt-0.10.2-29.el6_5.3.x86_64

OS version is the same, CentOS 6.4.

The cloud system is OpenNebula (4.4) and the migration fails also when the listen_vnc (different from 0.0.0.0) is setted directly in the template.