Bug 1126487

Summary: virsh and perl-Sys-Virt unable to update active running device interface
Product: Red Hat Enterprise Linux 6 Reporter: Jarrod Makin <jarrod.makin>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.6CC: berrange, dyuan, honzhang, jarrod.makin, mzhan, rbalakri
Target Milestone: rcKeywords: MoveUpstream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-49.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-22 05:46:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jarrod Makin 2014-08-04 14:27:55 UTC
Description of problem:
Using either:
virsh update-device <domain> interface-definition.xml
or perl-Sys-Virt's
$dom->update_device
methods, it isn't possible to change the interface definition in /var/run/libvirt/qemu/domain.xml or get the ebtables-rules updated.

This is a problem is vps users want to start using additional ip addresses whilst remaining online.

Version-Release number of selected component (if applicable):
libvirt-0.10.2-29.el6_5.11.x86_64

Does libvirt need an "update-interface" method for specific cases like this?

How reproducible:
Easy

Steps to Reproduce:

#virsh method
1. Make a small xml definition file, for example interface-definition.xml
<interface type="bridge">
  <bandwidth>
    <inbound average="12800" />
    <outbound average="12800" />
  </bandwidth>
  <filterref filter="clean-traffic">
    <parameter name="IP" value="1.2.3.4" />
    <parameter name="IP" value="1.2.3.5" />
  </filterref>
  <mac address="52:54:00:00:00:01" />
  <source bridge="br1" />
  <target dev="vps-1" />
</interface>
2. Check contents of /var/run/libvirt/qemu/domain.xml and /sbin/ebtables-save output.
3. virsh update-device domain interface-definition.xml
4. Compare contents of /var/run/libvirt/qemu/domain.xml and /sbin/ebtables-save output now

#perl method
perl -MSys::Virt -e "my \$vmm=Sys::Virt->new(); my \$dom=\$vmm->get_domain_by_name('domain'); \$dom->update_device(\"<interface type='bridge'>
  <bandwidth>
    <inbound average='12800' />
    <outbound average='12800' />
  </bandwidth>
  <filterref filter='clean-traffic'>
    <parameter name='IP' value='1.2.3.4' />
    <parameter name='IP' value='1.2.3.5' />
  </filterref>
  <mac address='52:54:00:00:00:01' />
  <source bridge='br1' />
  <target dev='vps-1' />
</interface>\")"

Actual results:
Timestamp on /var/run/libvirt/qemu/domain.xml is updated, but contents haven't changed.
ebtables rules don't change either.

Expected results:
Updated definition in /var/run/libvirt/qemu/domain.xml and updated ebtables rules

Additional info:
The update-device method is able to correctly update the persistent conf in /etc/libvirt/qemu/domain.xml.
Both the attach device and detach device methods work on the running interface, so I am detaching the interface and reattaching the new definition as a work around, although this is rather error prone.

Comment 3 Jarrod Makin 2014-10-29 11:29:27 UTC
I argue this is a bug rather than an enhancement request and confirm its presence in libvirt-0.10.2-46.el6_6.1.x86_64 and libvirt-client-0.10.2-46.el6_6.1.x86_64

Documentation for perl-Sys-Virt at http://search.cpan.org/~danberr/Sys-Virt-0.10.2/lib/Sys/Virt/Domain.pm#DEVICE_HOTPLUG_OPTIONS suggests it should be possible to update a running domain's devices not just the persistent configuration.

Comment 4 Daniel Berrangé 2014-12-08 09:38:46 UTC
Re-assigning to libvirt, since the perl API binding doesn't influence functional results of the API calls.

Comment 5 Michal Privoznik 2015-01-22 08:30:35 UTC
(In reply to Jarrod Makin from comment #3)
> I argue this is a bug rather than an enhancement request and confirm its
> presence in libvirt-0.10.2-46.el6_6.1.x86_64 and
> libvirt-client-0.10.2-46.el6_6.1.x86_64
> 
> Documentation for perl-Sys-Virt at
> http://search.cpan.org/~danberr/Sys-Virt-0.10.2/lib/Sys/Virt/Domain.
> pm#DEVICE_HOTPLUG_OPTIONS suggests it should be possible to update a running
> domain's devices not just the persistent configuration.

Well, if I try to update the device I get 'not supported' error message. Why it should be a bug fix rather than an enhancement then?

virsh # update-device test interface-definition.xml
error: Failed to update device from interface-definition.xml
error: this function is not supported by the connection driver: unable to change config on 'bridge' network type


BTW: what's the domain XML? The interface must be already present there.
BTW2: There's a workaround: detach and attach the interface again.

Comment 7 Jarrod Makin 2015-01-22 13:21:54 UTC
Hi Michal,

I hope I am able to answer some of your questions.

I have never received the 'not supported' error message.
Additionally, I am unable to follow the link in your second post as I cannot resolve post-office.corp.redhat.com.

By domain XML, I refer to either the persistent definition in /etc/libvirt/qemu or the currently active definition in /var/run/libvirt/qemu

I have tried the workaround and find it sometimes doesn't work on certain guest operating systems, particularly older Ubuntus and some Windows. Instead, the detach works but then the attach fails, leaving my guests without any network.

My intention is to be able to restrict my users to only be able to raise a certain selection of ip address, and change that on the fly with these commands rather than manually invoke ebtables.

The reason I think of it as a bug, is because sometimes attach-device fails and because update-device successfully updates the persistent definition, but makes no change to the currently active definition or change the ebtables rules.

The reason I think an enhancement request might be more suitable is because there is currently no {attach/detach/update} interface, only {attach/detach/update} device.

Comment 9 hongming 2015-03-02 09:38:42 UTC
Verify it as follows. The result is expected. 


# rpm -q libvirt
libvirt-0.10.2-49.el6.x86_64

# virsh start rhel6.5
Domain rhel6.5 started

# virsh dumpxml  rhel6.5|grep interface -A8
    <interface type='network'>
      <mac address='52:54:00:f5:c6:22'/>
      <source network='default'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <filterref filter='clean-traffic'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>


# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j libvirt-I-vnet0

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-o vnet0 -j libvirt-O-vnet0

Bridge chain: libvirt-I-vnet0, entries: 9, policy: ACCEPT
-j I-vnet0-mac
-p IPv4 -j I-vnet0-ipv4-ip
-p IPv4 -j ACCEPT 
-p ARP -j I-vnet0-arp-mac
-p ARP -j I-vnet0-arp-ip
-p ARP -j ACCEPT 
-p 0x8035 -j I-vnet0-rarp
-p 0x835 -j ACCEPT 
-j DROP 

Bridge chain: libvirt-O-vnet0, entries: 4, policy: ACCEPT
-p IPv4 -j O-vnet0-ipv4
-p ARP -j ACCEPT 
-p 0x8035 -j O-vnet0-rarp
-j DROP 

Bridge chain: I-vnet0-mac, entries: 2, policy: ACCEPT
-s 52:54:0:f5:c6:22 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-ipv4-ip, entries: 3, policy: ACCEPT
-p IPv4 --ip-src 0.0.0.0 --ip-proto udp -j RETURN 
-p IPv4 --ip-src 192.168.122.12 -j RETURN 
-j DROP 

Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT
-j ACCEPT 

Bridge chain: I-vnet0-arp-mac, entries: 2, policy: ACCEPT
-p ARP --arp-mac-src 52:54:0:f5:c6:22 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-arp-ip, entries: 2, policy: ACCEPT
-p ARP --arp-ip-src 192.168.122.12 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -s 52:54:0:f5:c6:22 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:f5:c6:22 --arp-mac-dst 52:54:0:f5:c6:22 -j ACCEPT 
-j DROP 

Bridge chain: O-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:f5:c6:22 --arp-mac-dst 52:54:0:f5:c6:22 -j ACCEPT 
-j DROP 



# cat interface.xml 
<interface type='network'>
<mac address='52:54:00:f5:c6:22'/>
<source network='default'/>
<target dev='vnet0'/>
<model type='virtio'/>
<filterref filter="clean-traffic">
<parameter name="IP" value="1.2.3.4" />
<parameter name="IP" value="1.2.3.5" />
</filterref>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>


# virsh update-device rhel6.5 interface.xml 
Device updated successfully


# virsh dumpxml rhel6.5|grep interface -A10
    <interface type='network'>
      <mac address='52:54:00:f5:c6:22'/>
      <source network='default'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <filterref filter='clean-traffic'>
        <parameter name='IP' value='1.2.3.4'/>
        <parameter name='IP' value='1.2.3.5'/>
      </filterref>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>


# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j libvirt-I-vnet0

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-o vnet0 -j libvirt-O-vnet0

Bridge chain: libvirt-I-vnet0, entries: 9, policy: ACCEPT
-j I-vnet0-mac
-p IPv4 -j I-vnet0-ipv4-ip
-p IPv4 -j ACCEPT 
-p ARP -j I-vnet0-arp-mac
-p ARP -j I-vnet0-arp-ip
-p ARP -j ACCEPT 
-p 0x8035 -j I-vnet0-rarp
-p 0x835 -j ACCEPT 
-j DROP 

Bridge chain: libvirt-O-vnet0, entries: 4, policy: ACCEPT
-p IPv4 -j O-vnet0-ipv4
-p ARP -j ACCEPT 
-p 0x8035 -j O-vnet0-rarp
-j DROP 

Bridge chain: I-vnet0-mac, entries: 2, policy: ACCEPT
-s 52:54:0:f5:c6:22 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-ipv4-ip, entries: 4, policy: ACCEPT
-p IPv4 --ip-src 0.0.0.0 --ip-proto udp -j RETURN 
-p IPv4 --ip-src 1.2.3.4 -j RETURN 
-p IPv4 --ip-src 1.2.3.5 -j RETURN 
-j DROP 

Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT
-j ACCEPT 

Bridge chain: I-vnet0-arp-mac, entries: 2, policy: ACCEPT
-p ARP --arp-mac-src 52:54:0:f5:c6:22 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-arp-ip, entries: 3, policy: ACCEPT
-p ARP --arp-ip-src 1.2.3.4 -j RETURN 
-p ARP --arp-ip-src 1.2.3.5 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -s 52:54:0:f5:c6:22 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:f5:c6:22 --arp-mac-dst 52:54:0:f5:c6:22 -j ACCEPT 
-j DROP 

Bridge chain: O-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:f5:c6:22 --arp-mac-dst 52:54:0:f5:c6:22 -j ACCEPT 
-j DROP

Comment 11 errata-xmlrpc 2015-07-22 05:46:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1252.html