Bug 1020356 - stray libvirt networks make vdsm fail to recognize management network on native interface with "Network defined withoutdevices" error of setupNetworks
Summary: stray libvirt networks make vdsm fail to recognize management network on nati...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.4.0
Assignee: Antoni Segura Puimedon
QA Contact: Martin Pavlik
URL:
Whiteboard: network
Depends On:
Blocks: 1044024 rhev3.4beta 1142926
TreeView+ depends on / blocked
 
Reported: 2013-10-17 13:59 UTC by David Jaša
Modified: 2016-02-10 19:48 UTC (History)
13 users (show)

Fixed In Version: ovirt-3.4.0-beta2
Doc Type: Bug Fix
Doc Text:
The Red Hat Enterprise Virtualization Manager now recognizes preconfigured networks in which the management network is bridgeless and untagged, making it unnecessary to drag and drop these networks to the bond interface in order to recognize them.
Clone Of:
: 1044024 (view as bug list)
Environment:
Last Closed: 2014-06-09 13:25:25 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:0504 0 normal SHIPPED_LIVE vdsm 3.4.0 bug fix and enhancement update 2014-06-09 17:21:35 UTC
oVirt gerrit 22354 0 None None None Never
oVirt gerrit 22436 0 None None None Never

Description David Jaša 2013-10-17 13:59:46 UTC
Description of problem:
I added to RHEV-M a host with pre-configured networks with management network on untagged interface. vdsm failed to recognize this interface as a management interface. In webadmin, when I launch "setup host networks", I can see "rhevm" as an unattached network". When I attach the management network to the interface in the webui, the action fails with "Illegal Network parameters".

vdsmd.log says:
Thread-41::DEBUG::2013-10-17 15:47:02,068::BindingXMLRPC::974::vds::(wrapper) client [10.34.73.33]::call setupNetworks with ({'rhevm': {'bonding': 'bond0', 'ipaddr': '10.34.73.74', 'gateway': '10.34.73.126', 'bridged': 'false', 'netmask': '255.255.255.192'}}, {}, {'connectivityCheck': 'true', 'connectivityTimeout': 120}) {}
Thread-42::DEBUG::2013-10-17 15:47:02,070::BindingXMLRPC::974::vds::(wrapper) client [10.34.73.33]::call ping with () {}
Thread-42::DEBUG::2013-10-17 15:47:02,071::BindingXMLRPC::981::vds::(wrapper) return ping with {'status': {'message': 'Done', 'code': 0}}
Thread-41::ERROR::2013-10-17 15:47:02,264::API::1262::vds::(setupNetworks) Network defined withoutdevices.
Traceback (most recent call last):
  File "/usr/share/vdsm/API.py", line 1260, in setupNetworks
    supervdsm.getProxy().setupNetworks(networks, bondings, options)
  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
    **kwargs)
  File "<string>", line 2, in setupNetworks
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
    raise convert_to_error(kind, result)
ConfigNetworkError: (21, 'Network defined withoutdevices.')
Thread-41::DEBUG::2013-10-17 15:47:02,266::BindingXMLRPC::981::vds::(wrapper) return setupNetworks with {'status': {'message': 'Network defined withoutdevices.', 'code': 21}}


My networks            VLAN-tagged     VM network    IP configuration
---------------------------------------------------------------------------
rhevm                  no              no            static (preconfigured)
vm_dynamic             180             yes           no
display                183             no            static (preconfigured)
display_wanem          184             no            static (preconfigured)


My network layout on host:

eth1                                   vm_dynamic
     \                              /
       >-- bond0 (10.34.73.74)  --< -  display
     /     bonding mode 4           \
eth2                                   display_wanem



Version-Release number of selected component (if applicable):
si18 / vdsm-4.13.0-0.2.beta1.el6ev.x86_64
si19 / vdsm-4.13.0-0.3.beta1.el6ev.x86_64

How reproducible:
always

Steps to Reproduce:
1. preconfigure a network on the host with management network bridgeless and untagged
2. add the host to RHEV
3.

Actual results:
network can not be made functional - the already working native network is not recognized as "rhevm" management network

Expected results:
rhev-m picks up such networking automatically

Additional info:
in 3.2, the network is not recognized right away either but when you drag and drop it to the bond interface, it gets successfully recognized --> Regression.

Comment 1 David Jaša 2013-10-17 14:26:15 UTC
I tried to work around the issue by:

1) marking the tagged networks as not required - and trying to refresh capabilities followed by manually attaching rhevm network to the bond
2) marking the tagged networks as not required and actually disabling them on the host (ifdown NETWORK) - refresh, attach
3) marking the tagged networks as not required, actually disabling them on the host and marking management network as a VM network - refresh, attach
4) everything as in 3) - remove host, add host

none of these do actually work, the error is the same all the time. VDSM log for 3) is:
Thread-21::DEBUG::2013-10-17 16:18:04,986::BindingXMLRPC::984::vds::(wrapper) client [10.34.73.33]::call setupNetworks with ({'rhevm': {'ipaddr': '10.34.73.74', 'netmask': '255.255.255.192', 'bonding': 'bond0', 'STP': 'no', 'bridged': 'true', 'gateway': '10.34.73.126'}}, {}, {'connectivityCheck': 'true', 'connectivityTimeout': 120}) {}
Thread-22::DEBUG::2013-10-17 16:18:04,988::BindingXMLRPC::984::vds::(wrapper) client [10.34.73.33]::call ping with () {}
Thread-22::DEBUG::2013-10-17 16:18:04,989::BindingXMLRPC::991::vds::(wrapper) return ping with {'status': {'message': 'Done', 'code': 0}}
Thread-21::ERROR::2013-10-17 16:18:05,100::API::1262::vds::(setupNetworks) Network defined withoutdevices.
Traceback (most recent call last):
  File "/usr/share/vdsm/API.py", line 1260, in setupNetworks
    supervdsm.getProxy().setupNetworks(networks, bondings, options)
  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
    **kwargs)
  File "<string>", line 2, in setupNetworks
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
    raise convert_to_error(kind, result)
ConfigNetworkError: (21, 'Network defined withoutdevices.')
Thread-21::DEBUG::2013-10-17 16:18:05,110::BindingXMLRPC::991::vds::(wrapper) return setupNetworks with {'status': {'message': 'Network defined withoutdevices.', 'code': 21}}

Comment 2 David Jaša 2013-10-17 15:54:05 UTC
I found out what happened - there were vdsm-* networks left in libvirt, vdsm-rhevm was then configured for bond0.182 device. This prevented picking up of actual network configuration or any configuration changes.

Comment 3 David Jaša 2013-10-18 09:25:55 UTC
Created attachment 813664 [details]
engine log

Comment 11 Dan Kenigsberg 2013-10-18 20:55:24 UTC
Could you reproduce the creation of these "stray vdsm-* networks"? The content of supervdsm.log is expected to be more interesting than that of vdsm.log.

Comment 12 David Jaša 2013-10-19 11:20:34 UTC
The stray networks were there from the previous setup that had management network on bond interface - from another host that has original setup:
# virsh -r net-dumpxml vdsm-rhevm
<network>
  <name>vdsm-rhevm</name>
  <uuid>UUID</uuid>
  <forward dev='bond0.182' mode='passthrough'>
    <interface dev='bond0.182'/>
  </forward>
</network>

This "vdsm-rhevm" libvirt network effectively prevented attaching of RHEV/vdsm "rhevm" network to interface bond0.

Comment 13 Dan Kenigsberg 2013-10-20 21:42:19 UTC
Could you attach the content of supervdsm.log?

Comment 19 Martin Pavlik 2014-02-10 16:06:00 UTC
works with 
oVirt Engine Version: 3.4.0-0.7.beta2.el6 
vdsm-4.14.1-3.el6.x86_64

Comment 22 errata-xmlrpc 2014-06-09 13:25:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0504.html


Note You need to log in before you can comment on or make changes to this bug.