Bug 864053

Summary: [RHEVM] [backend] [Host Networking] DC/CL 3.0 unexpected exception on VLAN removal
Product: Red Hat Enterprise Virtualization Manager Reporter: Martin Pavlik <mpavlik>
Component: ovirt-engineAssignee: lpeer <lpeer>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Martin Pavlik <mpavlik>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.1.0CC: dyasny, gklein, iheim, lpeer, masayag, Rhev-m-bugs, sgrinber, yeylon, ykaul
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: network
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-10-10 15:06:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
log_collector
none
screenshot 1 none

Description Martin Pavlik 2012-10-08 13:16:34 UTC 
Created attachment 623449 [details]
log_collector

Description of problem:

when trying to detach VLAN from host NIC error message with unexpected exception appears (see screenshot)

Version-Release number of selected component (if applicable):

Setup:
Red Hat Enterprise Virtualization Manager Version: '3.1.0-20.el6ev' 
Data center / Cluster 3.0

Host:
Red Hat Enterprise Virtualization Hypervisor release 6.3 (20120926.0.auto59)
vdsm-reg-4.9.6-37.0.el6_3.noarch
libvirt-0.9.10-21.el6_3.4.x86_64
qemu-kvm-rhev-0.12.1.2-2.295.el6_3.2.x86_64

How reproducible:
reproduced twice in row

Steps to Reproduce:
not exactly sure which steps are needed, describing whole procedure which I did
1.create a logical network with VLAN tagging
2.add network to host
3.create VM and attach VLAN to it, remove the network from VM
4.create template from VM and attach VLAN to it, remove VLAN
5.detach VLAN from the cluster, attach VLAN to cluster, detach VLAN from cluster
6.try to remove VLAN from host NIC
  
Actual results:
unexpected exception appears

Expected results:
VLAN network is detached properly

Additional info:

log collector attached

engine.log
2012-10-08 14:53:49,738 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (ajp-/127.0.0.1:8702-7) [5dd431a4] Command RemoveNetworkVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to RemoveNetworkVDS, error = Unexpected exception


this problem appeared when running network sanity test plan: https://tcms.engineering.redhat.com/plan/6491/rhevmnetworking-sanity-legacy
Comment 1 Martin Pavlik 2012-10-08 13:20:52 UTC 
Created attachment 623450 [details]
screenshot 1
Comment 3 Moti Asayag 2012-10-08 22:38:49 UTC 
Martin, please attach vdsm logs as well since the failure occurred on host side.
Comment 4 Martin Pavlik 2012-10-09 07:51:42 UTC 
Hi Moti,

logs are there, but log collector is hiding them well :). Please look into archive in attachment under this path

/mp-rhevm31-2012100814581349701085/tmp/logcollector/RHEVH-and-PostgreSQL-reports/10.34.66.81/
Comment 5 Martin Pavlik 2012-10-09 14:57:36 UTC 
It seems that problem is actually caused by vdsm


MainProcess|Thread-571::ERROR::2012-10-08 12:53:49,713::supervdsmServer::66::SuperVdsm.ServerCallback::(wrapper) Error in delNetwork
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer.py", line 64, in wrapper
  File "/usr/share/vdsm/supervdsmServer.py", line 104, in delNetwork
  File "/usr/share/vdsm/configNetwork.py", line 1070, in delNetwork
  File "/usr/share/vdsm/configNetwork.py", line 205, in removeLibvirtNetwork
  File "/usr/share/vdsm/configNetwork.py", line 200, in _removeNetwork
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1957, in undefine
libvirtError: cannot remove config file '/etc/libvirt/qemu/networks/vdsm-sw1.xml': Device or resource busy
Thread-571::ERROR::2012-10-08 12:53:49,713::BindingXMLRPC::883::vds::(wrapper) libvirt error
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 879, in wrapper
  File "/usr/share/vdsm/BindingXMLRPC.py", line 346, in delNetwork
  File "/usr/share/vdsm/API.py", line 1182, in delNetwork
  File "/usr/share/vdsm/supervdsm.py", line 59, in __call__
  File "/usr/share/vdsm/supervdsm.py", line 57, in <lambda>
  File "<string>", line 2, in delNetwork
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
libvirtError: cannot remove config file '/etc/libvirt/qemu/networks/vdsm-sw1.xml': Device or resource busy
Comment 6 Martin Pavlik 2012-10-10 15:06:08 UTC 
It seems that there is no reliable reproducer for this bug