Bug 1094205

Summary: virtual network static routes not checked for collision with existing routes/interfaces on host
Product: Red Hat Enterprise Linux 7 Reporter: Hu Jianwei <jiahu>
Component: libvirtAssignee: Martin Kletzander <mkletzan>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: low    
Version: 7.0CC: dyuan, honzhang, laine, mzhan, rbalakri
Target Milestone: rcKeywords: Upstream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.17-4.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 05:46:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hu Jianwei 2014-05-05 10:03:04 UTC
Description of problem:
Libvirt had better improve the process of route collision when having static route in virtual network

Version-Release number of selected component (if applicable):
libvirt-1.1.1-29.el7.x86_64
qemu-kvm-rhev-1.5.3-60.el7ev.x86_64
kernel-3.10.0-121.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare two virtual networks as below, the "default" have a <route> subelement, its address belongs to "default_001" network.
[root@localhost ~]# virsh net-dumpxml default
<network>
  <name>default</name>
  <uuid>5de056c8-568f-4a90-8862-0048150aa230</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:b5:c0:46'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
  <route address='192.168.222.0' prefix='24' gateway='192.168.122.1'/>
</network>

[root@localhost ~]# virsh net-dumpxml default_001
<network>
  <name>default_001</name>
  <uuid>f77423c8-6c10-44d1-8b3c-af774a1a1aa3</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr5' stp='on' delay='0' />
  <mac address='52:54:00:25:cf:91'/>
  <ip address='192.168.222.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.222.2' end='192.168.222.254' />
    </dhcp>
  </ip>
</network>

2. Start the "default" network first, two new rules were generated by libvirt.
[root@localhost ~]# virsh net-start default
Network default started

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
...
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.222.0   192.168.122.1   255.255.255.0   UG    1      0        0 virbr0

3. Then, start the "default_001".
[root@localhost ~]# virsh net-start default_001
error: Failed to start network default_001
error: internal error: Network is already in use by interface virbr0

Actual results:
As shown above, after step 2, one "192.168.222.0" rule has been created by "default" network, the "default_001" can not be started anymore because of networkCheckRouteCollision() function.

In turn, we start "default_001" before "default", two networks can be started normally, as below:

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.66.7.254     0.0.0.0         UG    0      0        0 eno1
10.66.4.0       0.0.0.0         255.255.252.0   U     0      0        0 eno1
10.68.5.26      10.66.7.254     255.255.255.255 UGH   1      0        0 eno1
[root@localhost ~]# virsh net-start default_001
Network default_001 started

[root@localhost ~]# virsh net-start default
Network default started

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.66.7.254     0.0.0.0         UG    0      0        0 eno1
10.66.4.0       0.0.0.0         255.255.252.0   U     0      0        0 eno1
10.68.5.26      10.66.7.254     255.255.255.255 UGH   1      0        0 eno1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.222.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr5
192.168.222.0   192.168.122.1   255.255.255.0   UG    1      0        0 virbr0

Expected results:
Since we have above workaround(change starting order) for such scenario, I think libvirt had better improve the process/workflow on route collision.

Comment 4 Laine Stump 2014-06-18 09:46:45 UTC
(just to clarify - the configuration that was used above is nonsensical, and should result in an error in both cases. The <route> element's purpose is to setup routes *from the host* to networks that are otherwise *unreachable* from the host (i.e. on a network that is hidden behind a virtual machine; in the example given for this BZ, it looks like an attempt was made to provide a route for the virtual machines on network "default" to reach machines on network "default_001"; this is incorrect usage of <route> )

libvirt currently checks for route collisions only for the network's own address range, but not for its static routes. networkCheckRouteCollision() should have its calling parameters modified to take an IP and prefix/netmask rather than a network name, then it should be called from networkAddRouteToBridge()

Comment 8 Martin Kletzander 2015-07-09 14:13:35 UTC
Patches proposed upstream:

https://www.redhat.com/archives/libvir-list/2015-July/msg00418.html

Comment 10 Martin Kletzander 2015-07-14 08:00:52 UTC
Fixed upstream by v1.2.17-104-g0e3ad241f3f4:

commit 0e3ad241f3f421c23bbe2cbe11a70f8e9184dc65
Author:     Martin Kletzander <mkletzan>
AuthorDate: Thu Jul 9 15:53:25 2015 +0200

    network: Add another collision check into networkCheckRouteCollision

Comment 13 hongming 2015-09-02 08:20:15 UTC
Verify it as follows.The result is expected.Move its status to VERIFIED.

# rpm -q libvirt
libvirt-1.2.17-6.el7.x86_64

# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              inactive   yes           yes
 test                 inactive   no            yes

# virsh net-dumpxml default
<network>
  <name>default</name>
  <uuid>06242766-3cab-4475-a794-1d655e228f14</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:c0:18:da'/>
  <bandwidth>
    <inbound average='10000' peak='50000' burst='10240'/>
    <outbound average='1280' peak='2560' burst='2560'/>
  </bandwidth>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
  <route address='192.168.222.0' prefix='24' gateway='192.168.122.1'/>
</network>

# virsh net-dumpxml test
<network>
  <name>test</name>
  <uuid>246e457b-42e5-497e-bd98-5a178a2a4a6f</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr5' stp='on' delay='0'/>
  <mac address='52:54:00:25:cf:91'/>
  <ip address='192.168.222.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.222.2' end='192.168.222.254'/>
    </dhcp>
  </ip>
</network>

# virsh net-start test
Network test started

# virsh net-start default
error: Failed to start network default
error: internal error: Route address '192.168.222.0' conflicts with IP address for 'virbr5'

# virsh net-destroy test
Network test destroyed

# virsh net-start default
Network default started

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.66.7.254     0.0.0.0         UG    0      0        0 eno1
10.66.4.0       0.0.0.0         255.255.252.0   U     0      0        0 eno1
10.72.17.5      10.66.7.254     255.255.255.255 UGH   100    0        0 eno1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.222.0   192.168.122.1   255.255.255.0   UG    1      0        0 virbr0

# virsh net-start test
error: Failed to start network test
error: internal error: Network is already in use by interface virbr0

Comment 15 errata-xmlrpc 2015-11-19 05:46:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html