Bug 918412
| Summary: | cannot plug VNIC with port mirroring enabled on running VM | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Martin Pavlik <mpavlik> | ||||||||||
| Component: | vdsm | Assignee: | Antoni Segura Puimedon <asegurap> | ||||||||||
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Martin Pavlik <mpavlik> | ||||||||||
| Severity: | high | Docs Contact: | |||||||||||
| Priority: | unspecified | ||||||||||||
| Version: | 3.2.0 | CC: | acathrow, bazulay, danken, gklein, hateya, iheim, jkt, lpeer, mpavlik, myakove, Rhev-m-bugs, yeylon | ||||||||||
| Target Milestone: | --- | ||||||||||||
| Target Release: | --- | ||||||||||||
| Hardware: | x86_64 | ||||||||||||
| OS: | Linux | ||||||||||||
| Whiteboard: | network | ||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
| Doc Text: | Story Points: | --- | |||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2013-09-02 11:32:48 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | Network | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Attachments: |
|
||||||||||||
Created attachment 705803 [details]
screenshot 1
when running another test I noticed that it is also not possible to add a new VNIC with port mirroring enabled it seems like a typo somewhere in code look at this log line: parse_action: bad value (17:mirred)! seems like there is "mirred" instead of mirrored 1) create new VM with no VNICs with OS suppurting hotplug 2) run the VM, wait until it is UP 3) create new VNIC, connected to rhevm, link up, plugged, port mirroring enabled full log collector attached as log_collector_2 2013-03-07 15:44:29,672 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] Failed in HotPlugNicVDS method 2013-03-07 15:44:29,672 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] Error code ACTIVATE_NIC_FAILED and error message VDSGenericException: VDSErrorException: Failed to HotPlugNicVDS, error = Cannot find device "if109" bad action parsing parse_action: bad value (17:mirred)! Illegal "action" 2013-03-07 15:44:29,672 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] Command org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugNicVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=49, mMessage=Cannot find device "if109" bad action parsing parse_action: bad value (17:mirred)! Illegal "action" ]] 2013-03-07 15:44:29,672 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] HostName = dell-05 2013-03-07 15:44:29,672 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] Command HotPlugNicVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotPlugNicVDS, error = Cannot find device "if109" bad action parsing parse_action: bad value (17:mirred)! Illegal "action" 2013-03-07 15:44:29,672 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugNicVDSCommand] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] FINISH, HotPlugNicVDSCommand, log id: 5e568eb3 2013-03-07 15:44:29,673 ERROR [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-13) [6fb8c6cc] Command org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotPlugNicVDS, error = Cannot find device "if109" bad action parsing parse_action: bad value (17:mirred)! Illegal "action" Created attachment 706655 [details]
log_collector_2
Seems like this is coming from vdsm see vdms.log attached as log_collector_2 , this problem also fails migration of VMs with port mirror enabled
MainProcess|Thread-117381::ERROR::2013-03-07 15:44:29,375::supervdsmServer::81::SuperVdsm.ServerCallback::(wrapper) Error in setPortMirroring
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer.py", line 79, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer.py", line 285, in setPortMirroring
tc.setPortMirroring(networkName, ifaceName)
File "/usr/share/vdsm/tc.py", line 72, in setPortMirroring
_addTarget(network, QDISC_INGRESS, target)
File "/usr/share/vdsm/tc.py", line 48, in _addTarget
filter_replace(network, parent, filt)
File "/usr/share/vdsm/tc.py", line 130, in filter_replace
_process_request(command)
File "/usr/share/vdsm/tc.py", line 100, in _process_request
raise TrafficControlException(retcode, err, command)
TrafficControlException: (1, 'Cannot find device "if109"\nbad action parsing\nparse_action: bad value (17:mirred)!\nIllegal "action"\n', ['/sbin/tc', 'filter', 'replace', 'dev', 'rhevm', 'parent', 'ffff:', 'prio', '49152', 'handle', '800::800', 'protocol', 'ip', 'u32', 'match', 'u8', '0', '0', 'action', 'mirred', 'egress', 'mirror', 'dev', 'if109', 'action', 'mirred', 'egress', 'mirror', 'dev', 'vnet1', 'action', 'mirred', 'egress', 'mirror', 'dev', u'vnet2'])
Martin, 'dev if109' is what 'tc' reports when if cannot find the source bridge. The interesting issue is how your system has got to this condition. Does this bug reproduce after the host is rebooted? How? The key it to understand how you get into /sbin/tc filter show dev rhevm parent ffff: reporting ifXXX devices. Dan, I probably found a reason of this behavior, this problem appears if you assign some network with display role, but you do not assign an IP to the host interface attached to this network. in steps: 1) create logical network (bridgeless in my case) and attach to your cluster 2) assign the network with display role 3) attach the network to host/hosts with IP set to none 4) try to run VM I would suggest some warning in GUI, that it is not recommended to use none IP in case of display network, or even block "none IP" option if display role is set. log collector attached as log_collector_3 GUI error: VM RHEL64 is down. Exit message: internal error ifname "vnet0" not in key map. vdsm.log: Thread-56058::DEBUG::2013-04-18 09:05:37,395::vm::676::vm.Vm::(_startUnderlyingVm) vmId=`e536ea57-36c4-40c3-bea7-5062299667a1`::_ongoingCreations released Thread-56058::ERROR::2013-04-18 09:05:37,395::vm::700::vm.Vm::(_startUnderlyingVm) vmId=`e536ea57-36c4-40c3-bea7-5062299667a1`::The vm start process failed Traceback (most recent call last): File "/usr/share/vdsm/vm.py", line 662, in _startUnderlyingVm File "/usr/share/vdsm/libvirtvm.py", line 1524, in _run File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 104, in wrapper File "/usr/lib64/python2.6/site-packages/libvirt.py", line 2645, in createXML libvirtError: internal error ifname "vnet0" not in key map Thread-56058::DEBUG::2013-04-18 09:05:37,402::vm::1064::vm.Vm::(setDownStatus) vmId=`e536ea57-36c4-40c3-bea7-5062299667a1`::Changed state to Down: internal error ifname "vnet0" not in key map Created attachment 737251 [details]
log_collector_3
Martin, the description of comment 6 is not in line with the bug summary. Do you currently experience any problem with port mirroring or the `tc` tool? If not, what you describe is a dup of bug 955429. Would you please either close this bug or supply a reproduction for the Cannot find device "if109" issue? |
Created attachment 705802 [details] sosreport-LogCollector-admin-20130306091226-6399.tar Description of problem: If VM is launched with unplugged VNIC which has port mirroring enabled, the VNIC cannot be plugged on running VM. Version-Release number of selected component (if applicable): Red Hat Enterprise Virtualization Manager Version: 3.2.0-10.10.beta1.el6ev How reproducible: 100% Steps to Reproduce: 1) Create VM - with 1 VNIC connected to rhevm, link up, port mirroring enabled, unplugged 2) Start VM, wait until VM is UP 3) go to VNIC edit dialog -> plug the VNIC Actual results: Error while executing action Edit VM Interface properties: Failed to activate VM Network Interface. Expected results: Interface activated Additional info: 2013-03-06 09:09:25,321 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-2) [2824eabf] Failed in HotPlugNicVDS method 2013-03-06 09:09:25,321 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-2) [2824eabf] Error code ACTIVATE_NIC_FAILED and error message VDSGenericException: VDSErrorException: Failed to HotPlugNicVDS, error = internal error ifname "vnet0" not in key map 2013-03-06 09:09:25,322 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-2) [2824eabf] Command org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugNicVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=49, mMessage=internal error ifname "vnet0" not in key map]] 2013-03-06 09:09:25,322 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (ajp-/127.0.0.1:8702-2) [2824eabf] HostName = dell-05 2013-03-06 09:09:25,322 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (ajp-/127.0.0.1:8702-2) [2824eabf] Command HotPlugNicVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotPlugNicVDS, error = internal error ifname "vnet0" not in key map 2013-03-06 09:09:25,322 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugNicVDSCommand] (ajp-/127.0.0.1:8702-2) [2824eabf] FINISH, HotPlugNicVDSCommand, log id: 597de01e 2013-03-06 09:09:25,322 ERROR [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-2) [2824eabf] Command org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotPlugNicVDS, error = internal error ifname "vnet0" not in key map