Bug 805071

We need the ability to

- Dynamically change then network that a vNic is attached to
eg. change from for  <interface type='bridge'>
      <source bridge='br0'/> to br1

The minimum required level of functionality is to change source bridges, but it would be advantageous if we can also change network type - eg. bridge -> network 

- Support having a no source device|bridge attached to a network
eg. something like an empty source element.
In this state the guest should see down link state.
(This may require qemu change)

Use cases:

Allow a user to disconnect a VM from the network without needing to unplug the nic.

Allow a user to change a VMs network - eg. moving from bridge0 to bridge1 without needing to unplug/plug the nic.

Question: Should we support an optional flag to change the link state - eg. flap it so the guest sees a link down/link up change or just keep the link up?

The following patch has been pushed upstream, and will be in release 0.9.11 of libvirt:

commit 2711ac87160d7ac7d550c57f4339e6c6749942fa
Author: Hendrik Schwartke <hendrik>
Date:   Wed Mar 28 15:11:09 2012 -0400

    qemu: support live change of the bridge used by a guest network device
    
It supports changing the bridge for interfaces of type='bridge'. It does not support changing networks, or changing from one type of interface to another.

(In reply to comment #2)
> Question: Should we support an optional flag to change the link state - eg.
> flap it so the guest sees a link down/link up change or just keep the link
> up?

I think it would be nice to have such an optional flag with the default "false".
true - link state is exposed to the guest
false - link state stays up all the time

I had originally proposed a patch series to upstream that used the qemu monitor netdev_del and netdev_add commands to completely remove the host side of a device and replace it with a new one, thus allowing more or less *any* change to the host side config of a network device:

  https://www.redhat.com/archives/libvir-list/2012-October/msg00542.html

But unfortunately this procedure, while completing successfully, doesn't actually work, as I detailed here in a message to qemu-devel:

  http://lists.nongnu.org/archive/html/qemu-devel/2012-10/msg02355.html

Because of that, I reposted an update PATCH 3/4 of the original series:

  https://www.redhat.com/archives/libvir-list/2012-October/msg00650.html

and it was that, along with PATCH 1/4 and 2/4 of the series, which were pushed upstream (i.e. the final patch of the original series was not posted at all).

The result is that you can now modify the host side connection of a network device as long as both the original and the new connection use a standard linux host bridge, i.e. as long as it is <interface type='bridge'>, or is <interface type='network', and the network uses "<bridge name='xxx'/>. Also note that Open vSwitch bridge connections are not currently supported.

Unfortunately, this means that switching between macvtap connections is not possible with virDomainUpdateDeviceFlags() - to do that, you will need to completely detach, then re-attach the device.

Upstream commits (with abridged commit logs):

commit 6bde0a1a37424c84492658223ff845b1ebb0e25c
Author: Laine Stump <laine>
Date:   Wed Oct 10 15:38:00 2012 -0400

    qemu: reorganize qemuDomainChangeNet and qemuDomainChangeNetBridge
    
    This patch resolves:
    
      https://bugzilla.redhat.com/show_bug.cgi?id=805071
    
    to the extent that it can be resolved with current qemu functionality.
    It attempts to detect as many situations as possible when the simple
    operation of disconnecting an existing tap device from one bridge and
    attaching it to another will satisfy the change requested in
    virDomainUpdateDeviceFlags() for a network device. Before this patch,
    that situation could only be detected if the pre-change interface
    *and* the post-change interface definition were both "type='bridge'".
    After this patch, it can also be detected if the before or after
    interfaces are any combination of type='bridge' and type='network'
    (the networks can be <forward mode='nat|route|bridge'>, as long as
    they use a Linux host bridge and not macvtap connections).
    
    Note that this function will refuse to make any change that requires
    the *guest* side of the device to be detached (e.g. changing the PCI
    address or mac address). Those would be disruptive enough to the guest
    that it's reasonable to require an explicit detach/attach sequence
    from the management application.
    
    This patch *does not* implement the "reconnect" code anyway - there is
    a placeholder that turns that into an error. Rather, the purpose of
    this patch is to replicate existing behavior with code that is ready
    to have that functionality plugged in in a later patch.


commit dc9d7a171c215efda1da17fca26a860030107fed
Author: Guido Günther <agx>
Date:   Mon Oct 15 09:58:28 2012 +0200

    Avoid straying </cpuset>
    
    by using the same condition as for the <cpuset>.
    
    Fixes "make check" found by
        http://honk.sigxcpu.org:8001/job/libvirt-check/160/

commit 11c47d979c885dec0799463cd9325e91fc4e46cc
Author: Laine Stump <laine>
Date:   Thu Oct 11 01:09:00 2012 -0400

    conf: virDomainDeviceInfoCopy utility function
    
    This does a shallow copy of all the bits, then strdups the two items
    that are actually allocated separately.

Hi, Laine
When I trying to update the network interface, i get following error:
libvirtd.log
------
2012-10-16 07:24:14.120+0000: 29942: error : qemuDomainChangeNet:1381 : this function is not supported by the connection driver: cannot find existing network device to modify
------

packages i used:
# rpm -q libvirt
libvirt-0.10.2-3.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.327.el6.x86_64

bridges in my host:
# ifconfig |grep virbr
virbr0    Link encap:Ethernet  HWaddr 52:54:00:EF:92:6A  
virbr1    Link encap:Ethernet  HWaddr 52:54:00:06:50:59  

interface in my guest:
 <interface type='bridge'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source bridge='virbr0'/>
      <target dev='vnet2'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>

# cat br1.xml 
<interface type='bridge'>
      <source bridge='virbr1'/>
      <model type='virtio'/>
</interface>

# virsh update-device mytest br1.xml 
error: Failed to update device from br1.xml
error: this function is not supported by the connection driver: cannot find existing network device to modify

You need to supply <mac address='xxxxx'/> - that is how it finds the original interface to modify.

Test with libvirt-0.10.2-3.el6.x86_64, and the bug can verified.
Test steps:
1. bridge+nat
1a) Prepare a guest with interface like:

 <interface type='bridge'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
</interface>
1b) Prepare a file for interface update:

# cat br1.xml 
<interface type='bridge'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source bridge='virbr1'/>
      <model type='virtio'/>
</interface>
1c) Start the guest, and check the guest's network work well, and check the guest's vnetX is connected to correct bridge
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.5254007da9f2       yes                  virbr0-nic
                                                                                   vnet0
virbr1          8000.525400682996       yes                  virbr1-nic
1d). update the guest's network(virbr0 -> virbr1)
# virsh update-device test1 br1.xml 
Device updated successfully

then 'service network restart ' in the guest, the guest get a new ip address belongs  to virbr1, and can ping out. Then check the guest's vnet0 is connected to the new bridge(virbr1)
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.5254007da9f2       yes                  virbr0-nic
virbr1          8000.525400682996       yes                 virbr1-nic
                                                                                   vnet0
2. network+nat
2a) Prepare a guest with interface like:

 <interface type='network'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source network='default'/>
      <model type='virtio'/>
</interface>

2b) Prepare a file for interface update:

  # cat new-net.xml 
<interface type='network'>
      <mac address='52:54:00:48:2e:f0'/>
      <source network='new-net'/>
      <model type='virtio'/>
</interface>

2c) Start the guest, and check the guest's network work well.

2d). update the guest's network (default -> new-net)
# virsh update-device test1 new-net.xml
Device updated successfully

then 'service network restart ' in the guest, the guest get a new ip address belongs  to virbr1, and can ping out.

3. network+route
3a) Prepare a new network with 'route' forward mode.

# virsh net-dumpxml net3
<network>
  <name>net3</name>
  <uuid>2c46e214-13fd-5ac8-cc49-a2d6e2c5cef3</uuid>
  <forward mode='route'/>
  <bridge name='virbr3' stp='on' delay='0' />
  <mac address='52:54:00:49:F4:7A'/>
  <ip address='192.168.124.1' netmask='255.255.255.252'>
    <dhcp>
      <range start='192.168.124.1' end='192.168.124.2' />
    </dhcp>
  </ip>
</network>
3b) Prepare  a guest and xml file like step 2a) and 2b), and start the guest
3c) update the guest's network (net3 -> net4 )
# virsh update-device test1 new-net.xml 
Device updated successfully
 
then 'service network restart ' in the guest, the guest get a new ip address  and can ping the host.  

4. bridge+route
steps like 1a)~1d).
4d) update the guest's network (virbr3 -> virbr4 )
# virsh update-device test1 br4.xml 
Device updated successfully

then 'service network restart ' in the guest, the guest get a new ip address  and can ping the host.  

5 Negative testing 
5.1 Try to change the <interface> definition to something that is <interface type='direct' ...>, and verify that "virsh update-device" logs an error.
5.1a) Prepare guest with a direct type interface, and change the source dev to prepare a file for network update
# virsh dumpxml test1
  <interface type='direct'>
      <mac address='52:54:00:48:2e:f0'/>
      <source dev='eth0' mode='bridge'/>
      <target dev='macvtap0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>
5.1b) Start the guest, and it can ping outside but fail ping host.
5.1c) Update the guest's network (eth0 -> eth1)
# virsh update-device test1 direct-net.xml 
error: Failed to update device from direct-net.xml
error: this function is not supported by the connection driver: unable to change config on 'direct' network type

6.2 Try to add a <bandwidth> section to the <interface> definition with virsh update-device, and verify that it again logs an error.
6.2a) Prepare a guest with bandwidth in the <interface> section, and edit the inbound average value to prepare a file for network update
# virsh dumpxml test1
 <interface type='network'>
      <mac address='52:54:00:48:2e:f0'/>
      <source network='default'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <bandwidth>
        <inbound average='2000' peak='6000' burst='5120'/>
        <outbound average='2000' peak='6000' burst='5120'/>
      </bandwidth>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>
6.2b) Start the guest, and its network works well..
6.2c) Update the guest's network (inbound average='2000'  -> inbound average='3000' )
# virsh update-device test1 bandwidth-net.xml
error: Failed to update device from bandwidth-net.xml
error: this function is not supported by the connection driver: unable to change config on 'network' network type

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html