Bug 1928078

Summary: Provide a way to ignore existing TC qdiscs and filters
Product: Red Hat Enterprise Linux 8 Reporter: Beniamino Galvani <bgalvani>
Component: NetworkManagerAssignee: Beniamino Galvani <bgalvani>
Status: CLOSED ERRATA QA Contact: Vladimir Benes <vbenes>
Severity: high Docs Contact:
Priority: high    
Version: 8.4CC: acardace, ahleihel, atragler, bgalvani, broose, egarver, fge, hakhande, lrintel, mleitner, rkhan, sukulkar, thaller, till, zshi
Target Milestone: rcKeywords: Triaged, ZStream
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: NetworkManager-1.32.0-0.5.el8 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 1972566 1972567 (view as bug list) Environment:
Last Closed: 2021-11-09 19:29:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1955160, 1972566, 1972567    
Attachments:
Description Flags
Reproducer for QE none

Description Beniamino Galvani 2021-02-12 10:41:27 UTC 
Currently, when a connection is activated NM always sets the kernel TC
configuration to what is described in the connection profile. In
particular if there is no 'tc' setting NM resets all the
qdiscs/filters added externally.

Since the TC support is very limited in NM and it will never implement
all the features supported by kernel, there should be a way to let NM
ignore the existing TC configuration.

We already do something similar for SR-IOV, where the absence of a
'sriov' settings means 'ignore', and a 'sriov' setting with num-vfs=0
means to clear everything on activation.
Comment 1 zenghui.shi 2021-04-07 02:02:32 UTC 
NM is used to activate the default route interface on OpenShift. 
When OVS HWOL is enabled on the default interface, this bug would
result in the tc filters (added by driver) be removed, which in
turn cause incomplete flows or packets be dropped unexpectedly.

It would be good to have a "ignore" setting in NM to leave the tc
filters (added by other components) untouched.
Comment 2 Marcelo Ricardo Leitner 2021-05-12 23:40:01 UTC 
Write up on integration issues noticed so far:
https://docs.google.com/document/d/13hP53xJu4LzBAT2o8mPfnumv7JnuUJx3pcmGghS195g/edit#
Comment 5 Alaa Hleihel (NVIDIA Mellanox) 2021-06-10 12:48:56 UTC 
Hi,

We (NVIDIA Mellanox) got many customer bugs about this issue.
Therefore, I'm requesting to z-stream the fix.

Thanks,
Alaa
Comment 6 Marcelo Ricardo Leitner 2021-06-10 21:16:07 UTC 
z-stream it back to 8.4, right? Is it enough?
Comment 7 Alaa Hleihel (NVIDIA Mellanox) 2021-06-13 11:46:13 UTC 
(In reply to Marcelo Ricardo Leitner from comment #6)
> z-stream it back to 8.4, right? Is it enough?

I think 8.2 is still widely used, right? so it will be great if we can get it there too.
Comment 8 Marcelo Ricardo Leitner 2021-06-15 23:00:37 UTC 
Gris, thoughts?
FWIW, I support this request to 8.2.z. This is a worth having fix back there as the issue is quite nasty to debug, unfortunately.
Comment 9 Gris Ge 2021-06-16 07:48:45 UTC 
Approving zstream for 8.4 and 8.2 after consulting with NetworkManager developer:

When there is no setting regarding tc filter and qdiscs in NetworkManager connection config, the default action regarding connection activation has changed:

From(old): Remove all existing tc filter and qdiscs.
To(new): Ignore existing tc filter and qdiscs.

Even this default action has been changed in zstream, but NetworkManager team agree this is the correct action and also be consistent with other settings like ethtool.
Meanwhile, the code change risk is acceptable.
Comment 12 Alaa Hleihel (NVIDIA Mellanox) 2021-06-16 08:30:42 UTC 
Thanks a lot!
Comment 13 Gris Ge 2021-06-17 12:25:03 UTC 
Hi Alaa Hlehel,

Could you use NetworkManager-1.32.0-0.5.el8 to test in your environment?

Thank you!
Comment 14 Alaa Hleihel (NVIDIA Mellanox) 2021-06-17 14:34:40 UTC 
Sure, the bug reprodueced with NetworkManager-1.30.0-7.el8.x86_64
the qdisc got removed after a while (when NM tried to configure the interface).

With NetworkManager-1.32.0-0.5.el8.x86_64 , the issue no longer reproduces, the tc ingress qdiscs are still there, they were not deleted even after NM tried to configure the interfaces.
(also checked using my old test https://bugzilla.redhat.com/attachment.cgi?id=1672300 )

I ran a couple of other OVS Offload tests and the traffic was offloaded without issue.

So it look good.
Thanks!
Comment 15 Beniamino Galvani 2021-06-17 16:07:00 UTC 
Created attachment 1791858 [details]
Reproducer for QE
Comment 16 Vladimir Benes 2021-06-18 13:44:45 UTC 
NMCI test case added:
https://gitlab.freedesktop.org/NetworkManager/NetworkManager-ci/-/merge_requests/784
Comment 17 Gris Ge 2021-09-17 09:52:51 UTC 
*** Bug 1753684 has been marked as a duplicate of this bug. ***
Comment 19 errata-xmlrpc 2021-11-09 19:29:43 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: NetworkManager security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4361