Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 919356

Summary: [RHEVM] [vdsm] unexpected exception on VNIC hot unplug with MAC change
Product: Red Hat Enterprise Virtualization Manager Reporter: Martin Pavlik <mpavlik>
Component: vdsmAssignee: Antoni Segura Puimedon <asegurap>
Status: CLOSED ERRATA QA Contact: Martin Pavlik <mpavlik>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.2.0CC: abaron, alkaplan, bazulay, danken, gklein, hateya, iheim, itzikb, lpeer, myakove, sgrinber, wudxw, ykaul, zdover
Target Milestone: ---   
Target Release: 3.2.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: network
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Previously, VDSM identified virtual network interfaces by their MAC addresses. This created a problem: if the user wanted to change the address at the same time it was unplugged, the virtual interface would not match. This meant that hot-unplugging a virtual interface and changing its MAC address at the same time raised an unexpected exception. Virtual network interfaces are now matched by alias, with the MAC address used as a fallback. Post-unplug hooks are no longer allowed to be executed when the virtual interface is not found. It is now possible to hot-unplug a virtual interface and change the MAC address in the same step.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-06-10 20:43:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
log_collector
none
screenshot 1 none

Description Martin Pavlik 2013-03-08 09:01:58 UTC
Created attachment 706945 [details]
log_collector

Description of problem:
when user tries to hotunplug VNIC and change its MAC address in one step, it raises unexpected exception

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Manager Version: 3.2.0-10.10.beta1.el6ev 
vdsm-4.10.2-10.0.el6ev.x86_64

How reproducible:
100%

Steps to Reproduce:
1) create VM with guest OS supporting hotplug, containing one VNIC connrcted to rhevm (active, plugged, port mirroring enabled)
2) Start VM wait until it is UP
3) Edit the VNIC -> unplugged, specify custom mac address (enter new valid MAC)
4) click OK
  
Actual results:
unexpected exception

Expected results:


Additional info:
engine.log
2013-03-08 09:52:12,830 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-4) [20389f6] Failed in HotUnplugNicVDS method
2013-03-08 09:52:12,831 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-4) [20389f6] Error code unexpected and error message VDSGenericException: VDSErrorException: Failed to HotUnplugNicVDS, error = Unexpected exception
2013-03-08 09:52:12,831 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (ajp-/127.0.0.1:8702-4) [20389f6] Command HotUnplugNicVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotUnplugNicVDS, error = Unexpected exception
2013-03-08 09:52:12,831 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HotUnplugNicVDSCommand] (ajp-/127.0.0.1:8702-4) [20389f6] FINISH, HotUnplugNicVDSCommand, log id: 710016f3
2013-03-08 09:52:12,831 ERROR [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-4) [20389f6] Command org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotUnplugNicVDS, error = Unexpected exception



vdsm.log
Thread-36994::ERROR::2013-03-08 09:52:12,819::libvirtvm::1714::vm.Vm::(hotunplugNic) vmId=`c89a9571-e5b0-4d00-923f-97648d6d1f19`::Hotunplug NIC failed - NIC not found: {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:cc', 'linkActive': 'true', 'network': 'rhevm', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '4e795622-b480-406b-b592-4b7ad662b71c', 'address': {'bus': '0x00', ' slot': '0x03', ' domain': '0x0000', ' type': 'pci', ' function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'portMirroring': ['rhevm']}
Thread-36994::ERROR::2013-03-08 09:52:12,826::BindingXMLRPC::932::vds::(wrapper) unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 918, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/BindingXMLRPC.py", line 263, in vmHotunplugNic
    return vm.hotunplugNic(params)
  File "/usr/share/vdsm/API.py", line 407, in hotunplugNic
    return curVm.hotunplugNic(params)
  File "/usr/share/vdsm/libvirtvm.py", line 1715, in hotunplugNic
    hooks.after_nic_hotunplug_fail(nicXml, self.conf)
UnboundLocalError: local variable 'nicXml' referenced before assignment

Comment 1 Martin Pavlik 2013-03-08 09:02:32 UTC
Created attachment 706946 [details]
screenshot 1

Comment 3 Dan Kenigsberg 2013-03-09 22:29:54 UTC
The real problem here is not the one solved by http://gerrit.ovirt.org/12891 - we need to find the relevant nic according to its unchanging "alias", not its mac address.

This applies with no regard to port mirroring.

Comment 4 itzikb 2013-03-10 21:09:23 UTC
Is it a valid case to change the MAC while unplugging the NIC?

Comment 5 Mark Wu 2013-03-11 09:25:26 UTC
I got the same question.  Shouldn't be disallowed on engine UI?
(In reply to comment #4)
> Is it a valid case to change the MAC while unplugging the NIC?

Comment 6 Dan Kenigsberg 2013-03-11 09:49:46 UTC
(In reply to comment #4)
> Is it a valid case to change the MAC while unplugging the NIC?

This bug has occurred because the MAC was changed in Engine before the hotunplug command was sent to vdsm. I believe that this has been recently changed by Alona's http://gerrit.ovirt.org/#/c/12792/ to solve bug 904840.

(In reply to comment #5)
> I got the same question.  Shouldn't be disallowed on engine UI?

I believe that this is HOW we change the VNIC mac address: we unplug it, change the MAC, and plug it back. I'll let Alona correct me if I'm wrong.

Comment 7 Alona Kaplan 2013-03-11 09:54:53 UTC
(In reply to comment #6)
> (In reply to comment #4)
> > Is it a valid case to change the MAC while unplugging the NIC?
> 
> This bug has occurred because the MAC was changed in Engine before the
> hotunplug command was sent to vdsm. I believe that this has been recently
> changed by Alona's http://gerrit.ovirt.org/#/c/12792/ to solve bug 904840.
> 
> (In reply to comment #5)
> > I got the same question.  Shouldn't be disallowed on engine UI?
> 
> I believe that this is HOW we change the VNIC mac address: we unplug it,
> change the MAC, and plug it back. I'll let Alona correct me if I'm wrong.

You are right. It is a valid case.

Comment 9 Meni Yakove 2013-03-24 08:06:29 UTC
Verified on:
vdsm-4.10.2-12.0.el6ev.x86_64
rhevm-3.2.0-10.16.master.el6ev.noarch

Comment 10 Cheryn Tan 2013-04-03 07:02:27 UTC
This bug is currently attached to errata RHBA-2012:14332. If this change is not to be documented in the text for this errata please either remove it from the errata, set the requires_doc_text flag to minus (-), or leave a "Doc Text" value of "--no tech note required" if you do not have permission to alter the flag.

Otherwise to aid in the development of relevant and accurate release documentation, please fill out the "Doc Text" field above with these four (4) pieces of information:

* Cause: What actions or circumstances cause this bug to present.

* Consequence: What happens when the bug presents.

* Fix: What was done to fix the bug.

* Result: What now happens when the actions or circumstances above occur. (NB: this is not the same as 'the bug doesn't present anymore')

Once filled out, please set the "Doc Type" field to the appropriate value for the type of change made and submit your edits to the bug.

For further details on the Cause, Consequence, Fix, Result format please refer to:

https://bugzilla.redhat.com/page.cgi?id=fields.html#cf_release_notes

Thanks in advance.

Comment 12 errata-xmlrpc 2013-06-10 20:43:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0886.html