Created attachment 498285 [details]
dumpxml_on_VM

Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

This feature was related to upstream:

https://www.redhat.com/archives/libvir-list/2011-May/msg00662.html

This is not a 3.1 feature this is a RHEV 2.2 feature, we could not regress this way from 2.2 to 3.0

Laine, I'm thinking that there should be overlap with the network abstraction here.

Dan K - if this is to work as suggested (by allowing the specification of a libvirt network name rather than IP address for the listen attribute), then two things must be satisfied:

1) It must always be valid to assume that the IP address required for the listen is associated with a libvirt <network> definition (so, for example, if the desired listen IP was on a physical interface that wasn't used for a libvirt virtual network, this wouldn't work).

2) either qemu must allow listening on multiple IP addresses (including IPv6 addresses), or we must limit the scope of the functionality to only listening on the first encountered IPv4 address in the network definition.

Also note that I hadn't originally considered including IP address info in the network XML for system bridges at all (since it's already configured somewhere else). Supporting the functionality described in this BZ would require either than I do that, or that libvirt learn the IP address(es) associated with the given bridge (or other physical interface), eg via virInterfaceGetXMLDesc(), which is available on RHEL and Fedora, but currently not on other platforms.

(1) means a major change in how vdsm does networking. Instead of creating bridges on its own, it would to define a libvirt <network> for each. But it sounds as a GOOD major change, so sure, it's acceptable to me.

(2) the semantics of how libvirt assigns an IP address per user of a <network> is a pure libvirt decision. It sounds fairly reasonable for qemu to listen on a single address from the <network> pool, allocated by libvirt in some kind of round robin algorithm.

I may not fully grasp you final comment.. In my opinion, defining <network> IP address when it is already deducible via the <network> bridge, is an infringement of the 11th commandment - thou shall not duplicate data. What if the two mismatch? I would not shed tears if this feature's availability depends on virInterfaceGetXMLDesc's.

This won't be in the same patchset as Bug 643947 (logical network switch abstraction), but should be simpler once that work is done.

As far as assigning the listen address in a round robin fashion, I'm not sure that makes sense, since it's not really accomplishing any load balancing (all of those addresses are on the same machine, and connected to the same physical NIC). 

I'm assuming that this listen address will only be IPv4? (a single network may have multiple IPv4 and IPv6 addresses associated with it)

Note that, in the case of libvirt's virtual networks, this address will likely be reachable only from the host itself. I'm assuming that will be acceptable? (I'm guessing you likely just won't use that type of network for these cases)

I agree with the final paragraph - we can just grab the current IP address from the kernel (via netcf, libnl, or directly with an ioctl; whatever is most convenient).

(In reply to comment #11)
> Note that, in the case of libvirt's virtual networks, this address will likely
> be reachable only from the host itself. I'm assuming that will be acceptable?
> (I'm guessing you likely just won't use that type of network for these cases)

I can live with anything that works for bridge-based network, where the bridge has externally-accessible IPv4 address.

How about this: if <graphics> has a listenNetwork='xxx' attribute, we will do the following:

1) if the network is <forward mode='route|nat'> (or no forward) we will listen on the 1st specified IPv4 address of the bridge device that's created for the network.

2) if the network is <forward mode='anything else'> we will learn the 1st configured ipv4 address of the device specified in "dev='yyy'" using an ioctl (or exec "ip addr show dev yyy"), and listen on that address.

I've posted a series of patches which provides this functionality to the libvirt upstream list:

https://www.redhat.com/archives/libvir-list/2011-July/msg00300.html

This patch series depends on the series for Bug 643947:

https://www.redhat.com/archives/libvir-list/2011-July/msg00149.html

It still needs review, but I have tested it for all three of 1) libvirt virtual networks, 2) host bridges, and 3) direct mode (macvtap) interfaces, and it works properly for all of them.

A more comprehensive solution to this problem was proposed by Dan Berrange:

  https://www.redhat.com/archives/libvir-list/2011-July/msg01636.html

I updated the last two patches in the series in Comment 16 and posted them to libvir-list, where they're awaiting review:

  http://www.redhat.com/archives/libvir-list/2011-July/msg01717.html


The other two patches from the series were already ACKed and pushed last week:

commit c5d1592e20899917b038463a77cd47eac2ce55a9
Author: Laine Stump <laine>
Date:   Thu Jul 7 00:15:08 2011 -0400

    util: add an ifaceGetIPAddress to the interface utilities

commit 239322cbd4d559231b1e6cfb6a71c6c884752026
Author: Laine Stump <laine>
Date:   Thu Jul 7 00:24:08 2011 -0400

    network: provide internal API to return IP of a network

The last two patches required to implement this functionality have been pushed to upstream libvirt. The syntax matches what was proposed in the email in comment 17.

(the third listed patch corrects an omission from the first):

commit ef79fb5b5f56e1995874d0c609da36edfa5acf0d
Author: Laine Stump <laine>
Date:   Thu Jul 7 00:20:28 2011 -0400

    conf: add <listen> subelement to domain <graphics> element

commit 99e4b30b39558d15e23f7c453551f7100c363244
Author: Laine Stump <laine>
Date:   Thu Jul 7 02:12:04 2011 -0400

    qemu: support type=network in domain graphics <listen>

commit eb1e3143da87101a829ebb4724c35c9b6361ca2b
Author: Laine Stump <laine>
Date:   Thu Jul 28 14:38:14 2011 -0400

    libxl: fix build failure due to change in virDomainGraphicsDef

verify pass on 3 types:
kernel-2.6.32-171.el6.x86_64
libvirt-0.9.4-0rc1.2.el6.x86_64
qemu-kvm-0.12.1.2-2.172.el6.x86_64

1. virtual network
on target 
# virsh net-dumpxml br1
<network>
  <name>br1</name>
  <uuid>25f0190f-4589-9430-2bb1-33e73ff61f43</uuid>
  <bridge name='br1' stp='on' delay='0' />
  <mac address='52:54:00:86:02:E8'/>
  <ip address='10.0.1.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.0.1.2' end='10.0.1.254' />
    </dhcp>
  </ip>
</network>
on source
# virsh net-dumpxml br1
<network>
  <name>br1</name>
  <uuid>e03baaeb-97a6-5ba4-0e14-59caf52fa021</uuid>
  <bridge name='br1' stp='on' delay='0' />
  <mac address='52:54:00:96:87:7C'/>
  <ip address='10.0.0.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.0.0.2' end='10.0.0.254' />
    </dhcp>
  </ip>
</network>
# virsh dumpxml guest
...
   <source network='br1'/>
...
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'>
      <listen type='network' network='br1'/>
    </graphics>
...
# virsh start guest
# virsh dumpxml guest
...
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.0.0.1' network='br1'/>
    </graphics>
...
can connect with #vncviewer 10.0.0.1:5900
# virsh migrate --live guest qemu+ssh://10.66.83.215/system
no error
on target 
# virsh dumpxml guest
...
    <graphics type='vnc' port='5901' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.0.1.1' network='br1'/>
    </graphics>
...
can connect with #vncviewer 10.0.1.1:5901

2. macvtap (type=direct)
on target && source
# virsh net-dumpxml direct-net
<network>
  <name>direct-net</name>
  <uuid>a109c4c7-8095-1945-37e3-eae0b6bc2af8</uuid>
  <forward dev='eth0' mode='bridge'>
    <interface dev='eth0'/>
  </forward>
</network>
on source
# virsh dumpxml guest
...
 <source network='direct-net'/>
...
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'>
      <listen type='network' network='direct-net'/>
    </graphics>
...
start guest and #virsh dumpxml guest
...
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.217' network='direct-net'/>
    </graphics>
...
can connect with vncviewer 10.66.83.217:5900
# virsh migrate --live guest qemu+ssh://10.66.83.215/system
no error
on target 
# virsh dumpxml guest
...
    <graphics type='vnc' port='5901' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.215' network='direct-net'/>
    </graphics>
...
can connect with vncviewer 10.66.83.215:5901

3. host bridge
first configure host bridge
here I use is breth0
on target && source
# virsh net-dumpxml red-network
<network>
  <name>red-network</name>
  <uuid>86424210-3a62-f2ce-f07f-ec29e412340d</uuid>
  <forward mode='bridge'/>
  <bridge name='breth0' />
</network>
on source
# virsh dumpxml guest
...
 <source network='red-network'/>
...
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'>
      <listen type='network' network='red-netowrk'/>
    </graphics>
...
start the guest and # virsh dumpxml guest
...
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.217' network='red-network'/>
    </graphics>
...
can connect with vncviewer 10.66.83.217:5900
# virsh migrate --live guest qemu+ssh://10.66.83.215/system
no error
on target 
# virsh dumpxml guest
...
    <graphics type='vnc' port='5901' autoport='yes' keymap='en-us'>
      <listen type='network' address='10.66.83.215' network='red-network'/>
    </graphics>
...
can connect with vncviewer 10.66.83.215:5901

Move the bug to verified according to comment 19.

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The address to be used for listening for VNC connections to a libvirt guest was previously required to be an IP address. In cases where the guest migrates from one host to another, and the admin wants the guest to be listening on a publicly visible interface, this address must necessarily change during migration.

In order to make this change in listen address possible, this RFE adds the option of specifying a "listen network" by name. libvirt will then locate the virtual network definition of that name on the current host, and use the IP address associated with that network (either the IP address of the bridge used by the network, or of the physical ethernet device specified by the network). By using the same network name on each host for a network definition using the device that should be used for vnc connection on that host, the guest can be migrated between the hosts, and its VNC listen address will change automatically as it migrates.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html