Bug 805071 - RFE : Dynamically change the host network/bridge that is attached to a vNIC
RFE : Dynamically change the host network/bridge that is attached to a vNIC
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.4
x86_64 Linux
high Severity high
: rc
: ---
Assigned To: Laine Stump
Virtualization Bugs
: FutureFeature
Depends On: 784767
Blocks: 782183 666891 787701 804821 840692 840699 853130 891059
  Show dependency treegraph
 
Reported: 2012-03-20 10:30 EDT by Andrew Cathrow
Modified: 2014-09-07 18:54 EDT (History)
20 users (show)

See Also:
Fixed In Version: libvirt-0.10.2-3.el6
Doc Type: Enhancement
Doc Text:
Feature: Change the bridge connected to a guest network device without detaching or disabling the guest or its network device. Reason: Sometimes an administrator may need to move a guest's network connection to a different network without stopping the guest. This functionality wasn't available in earlier versions of libvirt - in order to change the connection, the network device would need to be completely detached from the guest (making the guest believe it had been unplugged), then re-attached after changing the configuration to specify the new connection. Result: It is now possible (via "virsh update-device") to change a guest's <interface> definition to specify a different type (switching between "type='network'" and "type='bridge'"), and change the name of the bridge used for the given network, without stopping/pausing the guest or detaching its network device. From the point of view of the guest, the network will have remained up during this entire transition; if the move requires a new IP address, that must be handled by changing the configuration on the guest, or by requesting that it renew its dhcp lease.
Story Points: ---
Clone Of: 784767
: 853130 (view as bug list)
Environment:
Last Closed: 2013-02-21 02:08:53 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Andrew Cathrow 2012-03-20 10:30:00 EDT
+++ This bug was initially created as a clone of Bug #784767 +++

Created attachment 557594 [details]
Patch to implement functionality

Description of problem:


We would like to move the vNIC of a guest from one bridge to another while the guest is running. It is easy to do with brctl but libvirt does not provide an interface.

We have written a patch to support this feature using externally configured bridges. It currently does not support virtual networks generated by libvirt. We are working on that.

Attached:
- patch based on 6fba577e505611e6c25c68e322942eab7754de7e.
- example xml definition of guest
- example xml file for virsh updatedevice <guest>

--- Additional comment from ralf@spenneberg.net on 2012-01-26 02:42:18 EST ---

Created attachment 557595 [details]
Example xml definition of guest

--- Additional comment from ralf@spenneberg.net on 2012-01-26 02:43:39 EST ---

Created attachment 557596 [details]
File to use with virsh update-device

--- Additional comment from hendrik@os-t.de on 2012-01-26 07:39:49 EST ---

Created attachment 557667 [details]
fixed patch
Comment 2 Andrew Cathrow 2012-03-20 11:46:35 EDT
We need the ability to

- Dynamically change then network that a vNic is attached to
eg. change from for  <interface type='bridge'>
      <source bridge='br0'/> to br1

The minimum required level of functionality is to change source bridges, but it would be advantageous if we can also change network type - eg. bridge -> network 

- Support having a no source device|bridge attached to a network
eg. something like an empty source element.
In this state the guest should see down link state.
(This may require qemu change)

Use cases:

Allow a user to disconnect a VM from the network without needing to unplug the nic.

Allow a user to change a VMs network - eg. moving from bridge0 to bridge1 without needing to unplug/plug the nic.

Question: Should we support an optional flag to change the link state - eg. flap it so the guest sees a link down/link up change or just keep the link up?
Comment 3 Laine Stump 2012-03-30 20:21:13 EDT
The following patch has been pushed upstream, and will be in release 0.9.11 of libvirt:

commit 2711ac87160d7ac7d550c57f4339e6c6749942fa
Author: Hendrik Schwartke <hendrik@os-t.de>
Date:   Wed Mar 28 15:11:09 2012 -0400

    qemu: support live change of the bridge used by a guest network device
    
It supports changing the bridge for interfaces of type='bridge'. It does not support changing networks, or changing from one type of interface to another.
Comment 12 Ralf Spenneberg 2012-08-29 03:21:15 EDT
(In reply to comment #2)
> Question: Should we support an optional flag to change the link state - eg.
> flap it so the guest sees a link down/link up change or just keep the link
> up?

I think it would be nice to have such an optional flag with the default "false".
true - link state is exposed to the guest
false - link state stays up all the time
Comment 14 Laine Stump 2012-10-15 05:22:34 EDT
I had originally proposed a patch series to upstream that used the qemu monitor netdev_del and netdev_add commands to completely remove the host side of a device and replace it with a new one, thus allowing more or less *any* change to the host side config of a network device:

  https://www.redhat.com/archives/libvir-list/2012-October/msg00542.html

But unfortunately this procedure, while completing successfully, doesn't actually work, as I detailed here in a message to qemu-devel:

  http://lists.nongnu.org/archive/html/qemu-devel/2012-10/msg02355.html

Because of that, I reposted an update PATCH 3/4 of the original series:

  https://www.redhat.com/archives/libvir-list/2012-October/msg00650.html

and it was that, along with PATCH 1/4 and 2/4 of the series, which were pushed upstream (i.e. the final patch of the original series was not posted at all).

The result is that you can now modify the host side connection of a network device as long as both the original and the new connection use a standard linux host bridge, i.e. as long as it is <interface type='bridge'>, or is <interface type='network', and the network uses "<bridge name='xxx'/>. Also note that Open vSwitch bridge connections are not currently supported.

Unfortunately, this means that switching between macvtap connections is not possible with virDomainUpdateDeviceFlags() - to do that, you will need to completely detach, then re-attach the device.
Comment 15 Laine Stump 2012-10-15 05:28:32 EDT
Upstream commits (with abridged commit logs):

commit 6bde0a1a37424c84492658223ff845b1ebb0e25c
Author: Laine Stump <laine@laine.org>
Date:   Wed Oct 10 15:38:00 2012 -0400

    qemu: reorganize qemuDomainChangeNet and qemuDomainChangeNetBridge
    
    This patch resolves:
    
      https://bugzilla.redhat.com/show_bug.cgi?id=805071
    
    to the extent that it can be resolved with current qemu functionality.
    It attempts to detect as many situations as possible when the simple
    operation of disconnecting an existing tap device from one bridge and
    attaching it to another will satisfy the change requested in
    virDomainUpdateDeviceFlags() for a network device. Before this patch,
    that situation could only be detected if the pre-change interface
    *and* the post-change interface definition were both "type='bridge'".
    After this patch, it can also be detected if the before or after
    interfaces are any combination of type='bridge' and type='network'
    (the networks can be <forward mode='nat|route|bridge'>, as long as
    they use a Linux host bridge and not macvtap connections).
    
    Note that this function will refuse to make any change that requires
    the *guest* side of the device to be detached (e.g. changing the PCI
    address or mac address). Those would be disruptive enough to the guest
    that it's reasonable to require an explicit detach/attach sequence
    from the management application.
    
    This patch *does not* implement the "reconnect" code anyway - there is
    a placeholder that turns that into an error. Rather, the purpose of
    this patch is to replicate existing behavior with code that is ready
    to have that functionality plugged in in a later patch.


commit dc9d7a171c215efda1da17fca26a860030107fed
Author: Guido Günther <agx@sigxcpu.org>
Date:   Mon Oct 15 09:58:28 2012 +0200

    Avoid straying </cpuset>
    
    by using the same condition as for the <cpuset>.
    
    Fixes "make check" found by
        http://honk.sigxcpu.org:8001/job/libvirt-check/160/

commit 11c47d979c885dec0799463cd9325e91fc4e46cc
Author: Laine Stump <laine@laine.org>
Date:   Thu Oct 11 01:09:00 2012 -0400

    conf: virDomainDeviceInfoCopy utility function
    
    This does a shallow copy of all the bits, then strdups the two items
    that are actually allocated separately.
Comment 19 yanbing du 2012-10-16 03:36:09 EDT
Hi, Laine
When I trying to update the network interface, i get following error:
libvirtd.log
------
2012-10-16 07:24:14.120+0000: 29942: error : qemuDomainChangeNet:1381 : this function is not supported by the connection driver: cannot find existing network device to modify
------

packages i used:
# rpm -q libvirt
libvirt-0.10.2-3.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.327.el6.x86_64

bridges in my host:
# ifconfig |grep virbr
virbr0    Link encap:Ethernet  HWaddr 52:54:00:EF:92:6A  
virbr1    Link encap:Ethernet  HWaddr 52:54:00:06:50:59  

interface in my guest:
 <interface type='bridge'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source bridge='virbr0'/>
      <target dev='vnet2'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>

# cat br1.xml 
<interface type='bridge'>
      <source bridge='virbr1'/>
      <model type='virtio'/>
</interface>

# virsh update-device mytest br1.xml 
error: Failed to update device from br1.xml
error: this function is not supported by the connection driver: cannot find existing network device to modify
Comment 20 Laine Stump 2012-10-16 07:06:53 EDT
You need to supply <mac address='xxxxx'/> - that is how it finds the original interface to modify.
Comment 21 yanbing du 2012-10-18 02:55:19 EDT
Test with libvirt-0.10.2-3.el6.x86_64, and the bug can verified.
Test steps:
1. bridge+nat
1a) Prepare a guest with interface like:

 <interface type='bridge'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
</interface>
1b) Prepare a file for interface update:

# cat br1.xml 
<interface type='bridge'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source bridge='virbr1'/>
      <model type='virtio'/>
</interface>
1c) Start the guest, and check the guest's network work well, and check the guest's vnetX is connected to correct bridge
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.5254007da9f2       yes                  virbr0-nic
                                                                                   vnet0
virbr1          8000.525400682996       yes                  virbr1-nic
1d). update the guest's network(virbr0 -> virbr1)
# virsh update-device test1 br1.xml 
Device updated successfully

then 'service network restart ' in the guest, the guest get a new ip address belongs  to virbr1, and can ping out. Then check the guest's vnet0 is connected to the new bridge(virbr1)
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.5254007da9f2       yes                  virbr0-nic
virbr1          8000.525400682996       yes                 virbr1-nic
                                                                                   vnet0
2. network+nat
2a) Prepare a guest with interface like:

 <interface type='network'>
      <mac address='52:54:00:4a:c9:5e'/>
      <source network='default'/>
      <model type='virtio'/>
</interface>

2b) Prepare a file for interface update:

  # cat new-net.xml 
<interface type='network'>
      <mac address='52:54:00:48:2e:f0'/>
      <source network='new-net'/>
      <model type='virtio'/>
</interface>

2c) Start the guest, and check the guest's network work well.

2d). update the guest's network (default -> new-net)
# virsh update-device test1 new-net.xml
Device updated successfully

then 'service network restart ' in the guest, the guest get a new ip address belongs  to virbr1, and can ping out.

3. network+route
3a) Prepare a new network with 'route' forward mode.

# virsh net-dumpxml net3
<network>
  <name>net3</name>
  <uuid>2c46e214-13fd-5ac8-cc49-a2d6e2c5cef3</uuid>
  <forward mode='route'/>
  <bridge name='virbr3' stp='on' delay='0' />
  <mac address='52:54:00:49:F4:7A'/>
  <ip address='192.168.124.1' netmask='255.255.255.252'>
    <dhcp>
      <range start='192.168.124.1' end='192.168.124.2' />
    </dhcp>
  </ip>
</network>
3b) Prepare  a guest and xml file like step 2a) and 2b), and start the guest
3c) update the guest's network (net3 -> net4 )
# virsh update-device test1 new-net.xml 
Device updated successfully
 
then 'service network restart ' in the guest, the guest get a new ip address  and can ping the host.  

4. bridge+route
steps like 1a)~1d).
4d) update the guest's network (virbr3 -> virbr4 )
# virsh update-device test1 br4.xml 
Device updated successfully

then 'service network restart ' in the guest, the guest get a new ip address  and can ping the host.  

5 Negative testing 
5.1 Try to change the <interface> definition to something that is <interface type='direct' ...>, and verify that "virsh update-device" logs an error.
5.1a) Prepare guest with a direct type interface, and change the source dev to prepare a file for network update
# virsh dumpxml test1
  <interface type='direct'>
      <mac address='52:54:00:48:2e:f0'/>
      <source dev='eth0' mode='bridge'/>
      <target dev='macvtap0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>
5.1b) Start the guest, and it can ping outside but fail ping host.
5.1c) Update the guest's network (eth0 -> eth1)
# virsh update-device test1 direct-net.xml 
error: Failed to update device from direct-net.xml
error: this function is not supported by the connection driver: unable to change config on 'direct' network type

6.2 Try to add a <bandwidth> section to the <interface> definition with virsh update-device, and verify that it again logs an error.
6.2a) Prepare a guest with bandwidth in the <interface> section, and edit the inbound average value to prepare a file for network update
# virsh dumpxml test1
 <interface type='network'>
      <mac address='52:54:00:48:2e:f0'/>
      <source network='default'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <bandwidth>
        <inbound average='2000' peak='6000' burst='5120'/>
        <outbound average='2000' peak='6000' burst='5120'/>
      </bandwidth>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>
6.2b) Start the guest, and its network works well..
6.2c) Update the guest's network (inbound average='2000'  -> inbound average='3000' )
# virsh update-device test1 bandwidth-net.xml
error: Failed to update device from bandwidth-net.xml
error: this function is not supported by the connection driver: unable to change config on 'network' network type
Comment 22 errata-xmlrpc 2013-02-21 02:08:53 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html

Note You need to log in before you can comment on or make changes to this bug.