Bug 829037 - 3.1 - Several bridgeless networks on same NIC/Bond should be avoided
3.1 - Several bridgeless networks on same NIC/Bond should be avoided
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm (Show other bugs)
6.4
x86_64 Linux
medium Severity medium
: rc
: ---
Assigned To: Igor Lvovsky
Meni Yakove
network
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-05 16:08 EDT by Avi Tal
Modified: 2016-04-22 00:59 EDT (History)
13 users (show)

See Also:
Fixed In Version: vdsm-4.9.6-25.0
Doc Type: Bug Fix
Doc Text:
Previously, the VDSM was able to attach several bridgeless networks to the same interface. It did not clean up old bridgeless networks, which sometimes caused connection errors as the VDSM attempted to configure the obsolete network. Now, configurations are restricted to either a single bridged non-VLANed network or a single bridgeless non-VLANed network plus any amount of bridged/bridgeless VLANed networks. This prevents unnecessary connection errors caused by old or unused networks.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-12-04 13:58:51 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
ilvovsky: needinfo+


Attachments (Terms of Use)
engine log (11.69 KB, application/x-compressed-tar)
2012-06-05 16:08 EDT, Avi Tal
no flags Details
vdsm log (127.15 KB, application/x-compressed-tar)
2012-06-05 16:13 EDT, Avi Tal
no flags Details

  None (edit)
Description Avi Tal 2012-06-05 16:08:37 EDT
Created attachment 589630 [details]
engine log

Description of problem:
Bridgeless network are never being cleanup by VDSM.
using both SetupNetworks or the old Add network way to override an old bridgeless network over the same interface. VDSM will add the new network but won't remove the old bridgeless network. 
It will cause later problems when trying to reconfigure the old bridgeless network: ConfigNetworkError: (28, 'Bridge already exists').

I have added two bridgeless networks over the same interface (one after one. sw2 and sw3) then override this interface with bridged (sw4) network.

the results are:
[root@silver-vdsa ~]# virsh -r net-list --all
Name                 State      Autostart
-----------------------------------------
vdsm-rhevm           active     yes       
vdsm-sw2             active     yes       
vdsm-sw3             active     yes       
vdsm-sw4             active     yes       
default              inactive   no 

[root@silver-vdsa ~]# vdsClient 0 getVdsCaps |grep sw4 --color
        networks = {'rhevm': {'addr': '10.35.128.10', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.255.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.128.254', 'ports': ['eth0']}, 'sw3': {'interface': 'eth1', 'bridged': False}, 'sw2': {'interface': 'eth1', 'bridged': False}, 'sw4': {'addr': '', 'cfg': {'DEVICE': 'sw4', 'DELAY': '0', 'TYPE': 'Bridge', 'ONBOOT': 'yes', 'NM_CONTROLLED': 'no'}, 'mtu': '9000', 'netmask': '', 'stp': 'off', 'bridged': True, 'gateway': '0.0.0.0', 'ports': ['eth1']}}



As you can see, VDSM configured sw2, sw3 and sw4 over the same interface.

The Blocker issue is when trying to reconfigure the old bridgeless network:
MainProcess|Thread-16544::ERROR::2012-06-05 22:31:55,420::supervdsmServer::60::SuperVdsm.ServerCallback::(wrapper) Error in editNetwork
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer.py", line 58, in wrapper
    return func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer.py", line 102, in editNetwork
    return configNetwork.editNetwork(oldBridge, newBridge, **options)
  File "/usr/share/vdsm/configNetwork.py", line 783, in editNetwork
    addNetwork(newBridge, vlan=vlan, bonding=bonding, nics=nics, configWriter=configWriter, **options)
  File "/usr/share/vdsm/configNetwork.py", line 590, in addNetwork
    bridged=bridged, skipLibvirt=skipLibvirt)
  File "/usr/share/vdsm/configNetwork.py", line 503, in _addNetworkValidation
    raise ConfigNetworkError(ne.ERR_USED_BRIDGE, 'Bridge already exists')
ConfigNetworkError: (28, 'Bridge already exists')
Thread-16544::ERROR::2012-06-05 22:31:55,421::API::1209::vds::(editNetwork) Bridge already exists
Traceback (most recent call last):
  File "/usr/share/vdsm/API.py", line 1207, in editNetwork
    supervdsm.getProxy().editNetwork(oldBridge, newBridge, options)
  File "/usr/share/vdsm/supervdsm.py", line 59, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 57, in <lambda>
    callMethod = lambda : getattr(self._supervdsmProxy._svdsm, self._funcName)(*args, **kwargs)
  File "<string>", line 2, in editNetwork
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
    raise convert_to_error(kind, result)
ConfigNetworkError: (28, 'Bridge already exists')


BTW, running vdsClient delNetwork <bridgeless net> directly on the host will clean the network from libvirt.


Steps to Reproduce:
1. create bridgeless network and attach to host eth1
2. create bridged network nad attach to host eth1
3. try to reconfigure the old bridgeless network on eth1
  
Actual results:
libvirt will still contain the old bridgeless network
and the operation will fail due to 'Bridge already exists'

Expected results:
override bridgeless network should remove the network before adding a newer one on the same interface.

Additional info:
Comment 1 Avi Tal 2012-06-05 16:13:31 EDT
Created attachment 589631 [details]
vdsm log
Comment 3 Dan Kenigsberg 2012-06-05 17:38:13 EDT
(In reply to comment #0)

> Steps to Reproduce:
> 1. create bridgeless network and attach to host eth1
> 2. create bridged network nad attach to host eth1
> 3. try to reconfigure the old bridgeless network on eth1
>   
> Actual results:
> libvirt will still contain the old bridgeless network
> and the operation will fail due to 'Bridge already exists'
> 
> Expected results:
> override bridgeless network should remove the network before adding a newer
> one on the same interface.

I don't understand why this is a vdsm bug.

You create sw3 (bridgeless). You create sw4 (bridged) on the same eth1 nic. That's good. Now you try to make sw3 a bridged net. That's illegal, because you cannot connect one nic to two bridges.

Note that the operation that fails is
editNetwork with ('sw4', 'sw3', '', '', ['eth1'], {'STP': 'no'})
and not "reconfigure the old bridgeless". Vdsm is asked to reuse the name of an existing (bridged) network. That's illegal.

I haven't reviewed engine.log, maybe there's an issue there.

addNetwork with ('sw1', '162', '', ['eth1'], {'STP': 'no'}) {} flowID [429984de]
setSafeNetworkConfig with () {} flowID [1c5ac982]
delNetwork with ('sw1', '162', '', ['eth1']) {} flowID [307a7306]
setSafeNetworkConfig with () {} flowID [2b96155b]
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw3': {'ni
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw2': {'ni
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw2': {'re
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw2': {'ni
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw2': {'ni
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw2': {'ni
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw4': {'ni
editNetwork with ('sw4', 'sw4', '', '', ['eth1'], {'STP': 'no'}) {} flowID [24439552]
setSafeNetworkConfig with () {} flowID [2fd4cc7]
editNetwork with ('sw4', 'sw3', '', '', ['eth1'], {'STP': 'no'}) {} flowID [144cd1bc]
Comment 4 lpeer 2012-06-06 07:14:55 EDT
After talking with Avi the bug is as follows:
1. Add bridgeless network (red) to a nic (eth0), submit change
2. Remove red from eth0, submit change
3. Add (again) red network to eth0, submit change

Avi please confirm that this is what you inteded in the bug above.
Comment 5 Avi Tal 2012-06-06 07:32:47 EDT
(with danken) there *is* a vdsm bug here:

setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw3': {'nic': 'eth1', 'bridged': 'false'}}, {'bond4': {'remove': 'true'}, 'bond0': {'remove': 'true'}, 'bond1': {'remo
setupNetworks with ({'rhevm': {'nic': 'eth0', 'BOOTPROTO': 'dhcp', 'bridged': 'true'}, 'sw2': {'nic': 'eth1', 'bridged': 'false', 'mtu': '9000'}}, {'bond4': {'remove': 'true'}, 'bond0': {'remove': 'true'}, 

second operation should have failed since you cannot have two bridgeless networks on the same interface.
Comment 7 Avi Tal 2012-06-06 07:57:24 EDT
(In reply to comment #4)
> After talking with Avi the bug is as follows:
> 1. Add bridgeless network (red) to a nic (eth0), submit change
> 2. Remove red from eth0, submit change
> 3. Add (again) red network to eth0, submit change
> 
> Avi please confirm that this is what you inteded in the bug above.

Separate BZ was opened for backend: BZ#823927
Comment 9 Igor Lvovsky 2012-06-14 05:37:33 EDT
We can't reproduce this problem from UI or REST with latest engine.
The only way to reproduce it, is via vdsClient.
So, it can't be TestBlocker.
Comment 10 Igor Lvovsky 2012-07-18 10:37:34 EDT
http://gerrit.ovirt.org/#/c/6384/
Comment 12 Meni Yakove 2012-08-05 04:01:22 EDT
Verified on vdsm-4.9.6-26.0.el6_3.x86_64.
Comment 15 errata-xmlrpc 2012-12-04 13:58:51 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-1508.html

Note You need to log in before you can comment on or make changes to this bug.