Bug 703851 - LIBVIRT [RFE] [displayNetwork]- There should be a method to specify a symbolic network name, to be evaluated to IP on the destination host
LIBVIRT [RFE] [displayNetwork]- There should be a method to specify a symboli...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.1
Unspecified Unspecified
medium Severity high
: rc
: ---
Assigned To: Laine Stump
Virtualization Bugs
: FutureFeature
Depends On: 643947
Blocks: 693512 703848 753625
  Show dependency treegraph
 
Reported: 2011-05-11 08:59 EDT by Ortal
Modified: 2011-12-06 06:06 EST (History)
8 users (show)

See Also:
Fixed In Version: libvirt-0.9.4-0rc1.2.el6
Doc Type: Enhancement
Doc Text:
The address to be used for listening for VNC connections to a libvirt guest was previously required to be an IP address. In cases where the guest migrates from one host to another, and the admin wants the guest to be listening on a publicly visible interface, this address must necessarily change during migration. In order to make this change in listen address possible, this RFE adds the option of specifying a "listen network" by name. libvirt will then locate the virtual network definition of that name on the current host, and use the IP address associated with that network (either the IP address of the bridge used by the network, or of the physical ethernet device specified by the network). By using the same network name on each host for a network definition using the device that should be used for vnc connection on that host, the guest can be migrated between the hosts, and its VNC listen address will change automatically as it migrates.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-12-06 06:06:46 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Logs (888.34 KB, application/octet-stream)
2011-05-11 08:59 EDT, Ortal
no flags Details
dumpxml_on_VM (3.95 KB, text/plain)
2011-05-11 09:01 EDT, Ortal
no flags Details

  None (edit)
Description Ortal 2011-05-11 08:59:14 EDT
Created attachment 498284 [details]
Logs

LIBVIRT [RFE] [displayNetwork]- There should be a method to specify a symbolic network name, to be evaluated to IP on the destination host 


Prerequisites:

1 DC ver.2.3
2 Hosts in cluster (Intel conroe type)- RHEL 6.1
libvirt-0.8.7-18.el6.x86_64
vdsm-4.9-64.el6.x86_64
qemu-kvm-0.12.1.2-2.160.el6.x86_64


Scenario:

1. Create new VM with Windows OS (such as: Win-XP, Win7)
2. Use the default console for that VM (VNC)
3. Attempt to migrate that VM
4. migration failed, getting an error under libvirtd.log of:

libvirtd.log:

10:23:12.113: 2033: error : remoteIO:10569 : internal error process exited while connecting to monitor: Using CPU model "Conroe"
inet_listen_opts: bind(ipv4,10.35.104.15,5900): Cannot assign requested address
inet_listen_opts: FAILED

VDSM.log

Thread-37701::ERROR::2011-05-09 11:51:50,138::vm::330::vm.Vm::(_recover)
vmId=`a7df874f-e0a4-44cc-924a-5aebed0fca58`::internal error process exited
while connecting to monitor: Using CPU model "Conroe"
inet_listen_opts: bind(ipv4,10.35.104.15,5900): Cannot assign requested address
inet_listen_opts: FAILED

Thread-37704::DEBUG::2011-05-09 11:51:50,138::libvirtvm::197::vm.Vm::(run)
vmId=`a7df874f-e0a4-44cc-924a-5aebed0fca58`::migration downtime thread exiting
Thread-37701::ERROR::2011-05-09 11:51:50,723::vm::456::vm.Vm::(run)
vmId=`a7df874f-e0a4-44cc-924a-5aebed0fca58`::Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 447, in run
    self._startUnderlyingMigration()
  File "/usr/share/vdsm/libvirtvm.py", line 244, in _startUnderlyingMigration
    libvirt.VIR_MIGRATE_PEER2PEER, None, maxBandwidth)
  File "/usr/share/vdsm/libvirtvm.py", line 267, in f
    ret = attr(*args, **kwargs)
  File "/usr/share/vdsm/libvirtconnection.py", line 59, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 576, in
migrateToURI
    if ret == -1: raise libvirtError ('virDomainMigrateToURI() failed',
dom=self)
libvirtError: internal error process exited while connecting to monitor: Using
CPU model "Conroe"
inet_listen_opts: bind(ipv4,10.35.104.15,5900): Cannot assign requested address
inet_listen_opts: FAILED

____________

Thread-37701::ERROR::2011-05-09 11:51:50,138::vm::330::vm.Vm::(_recover)
vmId=`a7df874f-e0a4-44cc-924a-5aebed0fca58`::internal error process exited
while connecting to monitor: Using CPU model "Conroe"
inet_listen_opts: bind(ipv4,10.35.104.15,5900): Cannot assign requested address
inet_listen_opts: FAILED


dumpxml on the VM:

<graphics type='vnc' port='5900' autoport='yes' listen='10.35.102.21' keymap='en-us' passwdValidTo='1970-01-01T00:00:01'/>


** Please see attached VDSM.log & RHEVM.log
** Need to specify a simbolic network name, to be evaluated to IP on the destination host, something like
Comment 1 Ortal 2011-05-11 09:01:50 EDT
Created attachment 498285 [details]
dumpxml_on_VM
Comment 3 RHEL Product and Program Management 2011-05-12 02:01:14 EDT
Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.
Comment 4 Dan Kenigsberg 2011-05-12 05:14:40 EDT
This feature was related to upstream:

https://www.redhat.com/archives/libvir-list/2011-May/msg00662.html
Comment 7 Andrew Cathrow 2011-05-16 14:39:49 EDT
This is not a 3.1 feature this is a RHEV 2.2 feature, we could not regress this way from 2.2 to 3.0
Comment 8 Dave Allan 2011-05-17 10:39:56 EDT
Laine, I'm thinking that there should be overlap with the network abstraction here.
Comment 9 Laine Stump 2011-05-31 12:15:47 EDT
Dan K - if this is to work as suggested (by allowing the specification of a libvirt network name rather than IP address for the listen attribute), then two things must be satisfied:

1) It must always be valid to assume that the IP address required for the listen is associated with a libvirt <network> definition (so, for example, if the desired listen IP was on a physical interface that wasn't used for a libvirt virtual network, this wouldn't work).

2) either qemu must allow listening on multiple IP addresses (including IPv6 addresses), or we must limit the scope of the functionality to only listening on the first encountered IPv4 address in the network definition.

Also note that I hadn't originally considered including IP address info in the network XML for system bridges at all (since it's already configured somewhere else). Supporting the functionality described in this BZ would require either than I do that, or that libvirt learn the IP address(es) associated with the given bridge (or other physical interface), eg via virInterfaceGetXMLDesc(), which is available on RHEL and Fedora, but currently not on other platforms.
Comment 10 Dan Kenigsberg 2011-05-31 14:45:36 EDT
(1) means a major change in how vdsm does networking. Instead of creating bridges on its own, it would to define a libvirt <network> for each. But it sounds as a GOOD major change, so sure, it's acceptable to me.

(2) the semantics of how libvirt assigns an IP address per user of a <network> is a pure libvirt decision. It sounds fairly reasonable for qemu to listen on a single address from the <network> pool, allocated by libvirt in some kind of round robin algorithm.

I may not fully grasp you final comment.. In my opinion, defining <network> IP address when it is already deducible via the <network> bridge, is an infringement of the 11th commandment - thou shall not duplicate data. What if the two mismatch? I would not shed tears if this feature's availability depends on virInterfaceGetXMLDesc's.
Comment 11 Laine Stump 2011-06-07 02:36:16 EDT
This won't be in the same patchset as Bug 643947 (logical network switch abstraction), but should be simpler once that work is done.

As far as assigning the listen address in a round robin fashion, I'm not sure that makes sense, since it's not really accomplishing any load balancing (all of those addresses are on the same machine, and connected to the same physical NIC). 

I'm assuming that this listen address will only be IPv4? (a single network may have multiple IPv4 and IPv6 addresses associated with it)

Note that, in the case of libvirt's virtual networks, this address will likely be reachable only from the host itself. I'm assuming that will be acceptable? (I'm guessing you likely just won't use that type of network for these cases)

I agree with the final paragraph - we can just grab the current IP address from the kernel (via netcf, libnl, or directly with an ioctl; whatever is most convenient).
Comment 12 Dan Kenigsberg 2011-06-07 03:15:10 EDT
(In reply to comment #11)
> Note that, in the case of libvirt's virtual networks, this address will likely
> be reachable only from the host itself. I'm assuming that will be acceptable?
> (I'm guessing you likely just won't use that type of network for these cases)

I can live with anything that works for bridge-based network, where the bridge has externally-accessible IPv4 address.
Comment 13 Laine Stump 2011-06-12 20:53:24 EDT
How about this: if <graphics> has a listenNetwork='xxx' attribute, we will do the following:

1) if the network is <forward mode='route|nat'> (or no forward) we will listen on the 1st specified IPv4 address of the bridge device that's created for the network.

2) if the network is <forward mode='anything else'> we will learn the 1st configured ipv4 address of the device specified in "dev='yyy'" using an ioctl (or exec "ip addr show dev yyy"), and listen on that address.
Comment 16 Laine Stump 2011-07-07 03:42:18 EDT
I've posted a series of patches which provides this functionality to the libvirt upstream list:

https://www.redhat.com/archives/libvir-list/2011-July/msg00300.html

This patch series depends on the series for Bug 643947:

https://www.redhat.com/archives/libvir-list/2011-July/msg00149.html

It still needs review, but I have tested it for all three of 1) libvirt virtual networks, 2) host bridges, and 3) direct mode (macvtap) interfaces, and it works properly for all of them.
Comment 17 Laine Stump 2011-07-25 17:14:28 EDT
A more comprehensive solution to this problem was proposed by Dan Berrange:

  https://www.redhat.com/archives/libvir-list/2011-July/msg01636.html

I updated the last two patches in the series in Comment 16 and posted them to libvir-list, where they're awaiting review:

  http://www.redhat.com/archives/libvir-list/2011-July/msg01717.html


The other two patches from the series were already ACKed and pushed last week:

commit c5d1592e20899917b038463a77cd47eac2ce55a9
Author: Laine Stump <laine@laine.org>
Date:   Thu Jul 7 00:15:08 2011 -0400

    util: add an ifaceGetIPAddress to the interface utilities

commit 239322cbd4d559231b1e6cfb6a71c6c884752026
Author: Laine Stump <laine@laine.org>
Date:   Thu Jul 7 00:24:08 2011 -0400

    network: provide internal API to return IP of a network
Comment 18 Laine Stump 2011-07-29 13:44:43 EDT
The last two patches required to implement this functionality have been pushed to upstream libvirt. The syntax matches what was proposed in the email in comment 17.

(the third listed patch corrects an omission from the first):

commit ef79fb5b5f56e1995874d0c609da36edfa5acf0d
Author: Laine Stump <laine@laine.org>
Date:   Thu Jul 7 00:20:28 2011 -0400

    conf: add <listen> subelement to domain <graphics> element

commit 99e4b30b39558d15e23f7c453551f7100c363244
Author: Laine Stump <laine@laine.org>
Date:   Thu Jul 7 02:12:04 2011 -0400

    qemu: support type=network in domain graphics <listen>

commit eb1e3143da87101a829ebb4724c35c9b6361ca2b
Author: Laine Stump <laine@laine.org>
Date:   Thu Jul 28 14:38:14 2011 -0400

    libxl: fix build failure due to change in virDomainGraphicsDef
Comment 19 weizhang 2011-08-02 06:47:08 EDT
verify pass on 3 types:
kernel-2.6.32-171.el6.x86_64
libvirt-0.9.4-0rc1.2.el6.x86_64
qemu-kvm-0.12.1.2-2.172.el6.x86_64

1. virtual network
on target 
# virsh net-dumpxml br1
<network>
  <name>br1</name>
  <uuid>25f0190f-4589-9430-2bb1-33e73ff61f43</uuid>
  <bridge name='br1' stp='on' delay='0' />
  <mac address='52:54:00:86:02:E8'/>
  <ip address='10.0.1.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.0.1.2' end='10.0.1.254' />
    </dhcp>
  </ip>
</network>
on source
# virsh net-dumpxml br1
<network>
  <name>br1</name>
  <uuid>e03baaeb-97a6-5ba4-0e14-59caf52fa021</uuid>
  <bridge name='br1' stp='on' delay='0' />
  <mac address='52:54:00:96:87:7C'/>
  <ip address='10.0.0.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.0.0.2' end='10.0.0.254' />
    </dhcp>
  </ip>
</network>
# virsh dumpxml guest
...
   <source network='br1'/>
...
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'>
      <listen type='network' network='br1'/>
    </graphics>
...
# virsh start guest
# virsh dumpxml guest
...
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.0.0.1' network='br1'/>
    </graphics>
...
can connect with #vncviewer 10.0.0.1:5900
# virsh migrate --live guest qemu+ssh://10.66.83.215/system
no error
on target 
# virsh dumpxml guest
...
    <graphics type='vnc' port='5901' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.0.1.1' network='br1'/>
    </graphics>
...
can connect with #vncviewer 10.0.1.1:5901

2. macvtap (type=direct)
on target && source
# virsh net-dumpxml direct-net
<network>
  <name>direct-net</name>
  <uuid>a109c4c7-8095-1945-37e3-eae0b6bc2af8</uuid>
  <forward dev='eth0' mode='bridge'>
    <interface dev='eth0'/>
  </forward>
</network>
on source
# virsh dumpxml guest
...
 <source network='direct-net'/>
...
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'>
      <listen type='network' network='direct-net'/>
    </graphics>
...
start guest and #virsh dumpxml guest
...
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.217' network='direct-net'/>
    </graphics>
...
can connect with vncviewer 10.66.83.217:5900
# virsh migrate --live guest qemu+ssh://10.66.83.215/system
no error
on target 
# virsh dumpxml guest
...
    <graphics type='vnc' port='5901' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.215' network='direct-net'/>
    </graphics>
...
can connect with vncviewer 10.66.83.215:5901

3. host bridge
first configure host bridge
here I use is breth0
on target && source
# virsh net-dumpxml red-network
<network>
  <name>red-network</name>
  <uuid>86424210-3a62-f2ce-f07f-ec29e412340d</uuid>
  <forward mode='bridge'/>
  <bridge name='breth0' />
</network>
on source
# virsh dumpxml guest
...
 <source network='red-network'/>
...
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'>
      <listen type='network' network='red-netowrk'/>
    </graphics>
...
start the guest and # virsh dumpxml guest
...
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.217' network='red-network'/>
    </graphics>
...
can connect with vncviewer 10.66.83.217:5900
# virsh migrate --live guest qemu+ssh://10.66.83.215/system
no error
on target 
# virsh dumpxml guest
...
    <graphics type='vnc' port='5901' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.215' network='red-network'/>
    </graphics>
...
can connect with vncviewer 10.66.83.215:5901
Comment 21 dyuan 2011-08-03 03:35:19 EDT
Move the bug to verified according to comment 19.
Comment 22 Laine Stump 2011-11-16 11:55:13 EST
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The address to be used for listening for VNC connections to a libvirt guest was previously required to be an IP address. In cases where the guest migrates from one host to another, and the admin wants the guest to be listening on a publicly visible interface, this address must necessarily change during migration.

In order to make this change in listen address possible, this RFE adds the option of specifying a "listen network" by name. libvirt will then locate the virtual network definition of that name on the current host, and use the IP address associated with that network (either the IP address of the bridge used by the network, or of the physical ethernet device specified by the network). By using the same network name on each host for a network definition using the device that should be used for vnc connection on that host, the guest can be migrated between the hosts, and its VNC listen address will change automatically as it migrates.
Comment 23 errata-xmlrpc 2011-12-06 06:06:46 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html

Note You need to log in before you can comment on or make changes to this bug.