Bug 1730264 - VMs will fail to start if the vnic profile attached is having port mirroring enabled and have name greater than 15 characters
Summary: VMs will fail to start if the vnic profile attached is having port mirroring ...
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.4
Hardware: All
OS: Linux
medium
medium
Target Milestone: ovirt-4.4.0
: 4.4.0
Assignee: Ales Musil
QA Contact: Michael Burman
URL:
Whiteboard:
Depends On: 1765018
Blocks: 1744571
TreeView+ depends on / blocked
 
Reported: 2019-07-16 11:08 UTC by nijin ashok
Modified: 2020-01-24 19:49 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1744571 (view as bug list)
Environment:
Last Closed:
oVirt Team: Network
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
oVirt gerrit 102434 'None' MERGED core: Fix port mirror metadata for vm 2020-02-26 05:35:30 UTC
oVirt gerrit 102464 'None' MERGED core: Fix port mirror metadata for vm 2020-02-26 05:35:30 UTC

Description nijin ashok 2019-07-16 11:08:32 UTC
Description of problem:

If the network name is greater than 15 characters, the original name of the network will be  mapped to a different name while creating the bridge.


====
engine=# select name,vdsm_name from network where name='verylongnetworkname';
        name         |    vdsm_name    
---------------------+-----------------
 verylongnetworkname | on5604266439c64
(1 row)
====

However if the port mirroring is enabled, the vdsm still send setPortMirroring request to supervdsm with the "network name" instead of "vdsm_name". Because of this, the VM will fail to start with the error below in vdsm log since the bridge name "verylongnetworkname" doesn't exist.

====
2019-07-16 16:21:17,460+0530 ERROR (vm/b532b3ee) [virt.vm] (vmId='b532b3ee-c5f5-45e5-a1b9-136a7804b04b') The vm start process failed (vm:933)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 867, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2891, in _run
    self._domDependentInit()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2466, in _domDependentInit
    self._vmDependentInit()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2487, in _vmDependentInit
    supervdsm.getProxy().setPortMirroring(network, nic.name)
  File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__
    return callMethod()
  File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>
    **kwargs)
  File "<string>", line 2, in setPortMirroring
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
TrafficControlException: (1, 'Cannot find device "verylongnetwork"\n', ['/sbin/tc', 'qdisc', 'add', 'dev', 'verylongnetworkname', 'ingress'])
=====

Supervdsm log shows error below.

====
MainProcess|vm/b532b3ee::ERROR::2019-07-16 16:21:17,459::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper) Error in setPortMirroring
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 101, in wrapper
    res = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/network/tc/__init__.py", line 80, in setPortMirroring
    _qdisc_replace_ingress(network)
  File "/usr/lib/python2.7/site-packages/vdsm/network/tc/__init__.py", line 116, in _qdisc_replace_ingress
    qdisc.add(dev, 'ingress')
  File "/usr/lib/python2.7/site-packages/vdsm/network/tc/qdisc.py", line 43, in add
    _wrapper.process_request(command)
  File "/usr/lib/python2.7/site-packages/vdsm/network/tc/_wrapper.py", line 43, in process_request
    raise TrafficControlException(retcode, err, command)
TrafficControlException: (1, 'Cannot find device "verylongnetwork"\n', ['/sbin/tc', 'qdisc', 'add', 'dev', 'verylongnetworkname', 'ingress'])
====

Version-Release number of selected component (if applicable):

vdsm-4.30.17-1.el7ev.x86_64
rhvm-4.3.4.3-0.1.el7.noarch

How reproducible:

100%

Steps to Reproduce:

1. Create a network with name greater than 15 characters.
2. Enable port mirroring.
3. Attach this network to the VM and start it.

Actual results:

VMs will fail to start if the vnic profile attached is having port mirroring enabled and have name greater than 15 characters

Expected results:

VM should start.

Additional info:

Comment 2 Michael Burman 2019-07-23 06:55:09 UTC
QE managed to reproduce.
Edy, can you please take a look? is it vdsm to handle or is it engine side? 

Thanks,

Comment 3 Edward Haas 2019-07-24 10:26:42 UTC
(In reply to Michael Burman from comment #2)
> QE managed to reproduce.
> Edy, can you please take a look? is it vdsm to handle or is it engine side? 
> 
> Thanks,

Looks like Engine side. The long-name should never reach VDSM in the current solution.

Comment 8 Daniel Gur 2019-08-28 13:14:33 UTC
sync2jira

Comment 9 Daniel Gur 2019-08-28 13:19:35 UTC
sync2jira

Comment 11 Michael Burman 2019-10-23 12:02:05 UTC
I have no idea how this is possible, but the issue reproduced on rhv-4.4.0
4.4.0-0.4.master.el7
vdsm-4.40.0-127.gitc628cce.el8ev.x86_64
rhel8.1

MainProcess|vm/f1473d49::ERROR::2019-10-23 07:43:13,779::supervdsm_server::96::SuperVdsm.ServerCallback::(wrapper) Error in setPortMirroring
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 94, in wrapper
    res = func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py", line 83, in setPortMirroring
    _qdisc_replace_ingress(network)
  File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py", line 119, in _qdisc_replace_ingress
    qdisc.add(dev, 'ingress')
  File "/usr/lib/python3.6/site-packages/vdsm/network/tc/qdisc.py", line 43, in add
    _wrapper.process_request(command)
  File "/usr/lib/python3.6/site-packages/vdsm/network/tc/_wrapper.py", line 44, in process_request
    raise TrafficControlException(retcode, err, command)
vdsm.network.tc._wrapper.TrafficControlException: (2, 'Error: Exclusivity flag on, cannot modify.\n', ['/sbin/tc', 'qdisc', 'add', 'dev', 'ond0985e33f0f94', 'ingress'])
MainProcess|jsonrpc/0::DEBUG::2019-10-23 07:43:13,789::supervdsm_server::92::SuperVdsm.ServerCallback::(wrapper) call unsetPortMirroring with ('ond0985e33f0f94', 'vnet1') {}

Comment 12 Dominik Holler 2019-10-23 15:39:22 UTC
This is a new issue, probably rhel8 related:
the initial reported error was:
TrafficControlException: (1, 'Cannot find device "verylongnetwork"\n', ['/sbin/tc', 'qdisc', 'add', 'dev', 'verylongnetworkname', 'ingress'])
the new error is:
TrafficControlException: (2, 'Error: Exclusivity flag on, cannot modify.\n', ['/sbin/tc', 'qdisc', 'add', 'dev', 'ond0985e33f0f94', 'ingress'])

But never the less we should fix this.

Comment 13 Dominik Holler 2019-10-23 15:58:53 UTC
Networks with short names are affected, too.

Comment 14 Ales Musil 2019-10-24 06:25:44 UTC
That is different issue and IIRC we had discussion some time ago that tc is not working as vdsm expects.

Comment 15 Michael Burman 2019-10-24 07:10:53 UTC
(In reply to Ales Musil from comment #14)
> That is different issue and IIRC we had discussion some time ago that tc is
> not working as vdsm expects.

ack
Do you want new bug for that which will block this one?

Comment 16 Ales Musil 2019-10-24 07:14:30 UTC
Yes please, to keep it separate.

Comment 18 RHV Bug Bot 2019-12-13 13:15:59 UTC
WARN: Bug status (ON_QA) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.comINFO: Bug status (ON_QA) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.com

Comment 19 Michael Burman 2019-12-16 06:41:58 UTC
Verified on - vdsm-4.40.0-164.git38a19bb.el8ev.x86_64 with rhvm-4.4.0-0.9.master.el7.noarch

Comment 20 RHV Bug Bot 2019-12-20 17:45:31 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.comINFO: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.com

Comment 21 RHV Bug Bot 2020-01-08 14:47:47 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.comINFO: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.com

Comment 22 RHV Bug Bot 2020-01-08 15:17:24 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.comINFO: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.com

Comment 23 RHV Bug Bot 2020-01-24 19:49:30 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.comINFO: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops@redhat.com


Note You need to log in before you can comment on or make changes to this bug.