Bug 2003193 - Kubelet/crio leaks netns and veth ports in the host
Summary: Kubelet/crio leaks netns and veth ports in the host
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: 4.10.0
Assignee: Peter Hunt
QA Contact: Sunil Choudhary
URL:
Whiteboard: perfscale-ovn
: 2025329 (view as bug list)
Depends On:
Blocks: 2012836 2026386 2026388 2028126 2028127 2078400
TreeView+ depends on / blocked
 
Reported: 2021-09-10 14:52 UTC by Tim Rozet
Modified: 2023-09-15 01:14 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Pod namespaces (network, IPC, UTS) managed by CRI-O were only unmounted when the pod was removed Consequence: It seemed as though the namespaces were being leaked, as the kubelet takes a long time to remove pods Fix: Unmount and remove the namespaces on pod stop Result: It does not appear as though the namespaces are leaked
Clone Of:
: 2026386 2028126 2028127 (view as bug list)
Environment:
Last Closed: 2022-03-10 16:09:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github cri-o cri-o pull 5336 0 None Merged server: close namespaces on sandbox stop 2022-06-08 14:31:05 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:09:58 UTC

Description Tim Rozet 2021-09-10 14:52:44 UTC
Description of problem:
With openshift-sdn and OVN, veth ports are used to connect containers to OVS. The CNI will delete the OVS side of the veth port, while kubelet/crio is supposed to garbage collect and clean up leftover netns/veths after the pod is deleted. However, this is not happening and it causes the number of leaked veths to build up in the system over time. Consequently this drives OVS CPU to 100%, because OVS will iterate over all the ports in the host during certain events:

https://github.com/openvswitch/ovs/blob/f686957c9667ae962fb8fc003be2a5482e380d75/lib/netdev.c#L2191

The end result is over time on a node, pod latency and other performance impacts will occur due to these leaked ports.

Comment 1 Peter Hunt 2021-09-10 14:55:35 UTC
can you describe a concise reproducer so we can observe the netns/veths not being cleaned up?

Comment 2 Tim Rozet 2021-09-10 15:00:15 UTC
With 4.9, we can reproduce by running the node-density-lite scale test on a 20 node aws cluster repeatedly. In this case OVS has around 200 ports, but the host has over 2100 leftover netns and veths. I can reproduce it for you if you want. I think Dan Winship is going to add some more information to this bz with what he has found as well.

Comment 3 Dan Winship 2021-09-10 15:01:52 UTC
You don't need to do any scale stuff. We only *noticed* it at scale, but it will happen if you just create one pod and then delete it.

Comment 4 Tim Rozet 2021-09-10 15:02:14 UTC
the node-density-lite test will:
1. create 249 pods per node total, at a pod creation rate of 20/sec in a test namespace
2. after the test is complete, delete the namespace
3. re-run steps 1 and 2 multiple times

Comment 5 Dan Winship 2021-09-10 15:02:49 UTC
oh, and it appears to have started in 4.8. Earlier releases cleaned everything up properly.

Comment 6 Tim Rozet 2021-09-10 15:40:49 UTC
Filed https://bugzilla.redhat.com/show_bug.cgi?id=2003195 for OVN to ensure the host veths are removed on CNI delete or add failure.

Comment 8 Dan Winship 2021-09-10 16:56:00 UTC
(In reply to Dan Winship from comment #5)
> oh, and it appears to have started in 4.8. Earlier releases cleaned
> everything up properly.

Sorry, I screwed up my testing before. 4.7 has the bug too. So my test results are:

4.4 nightly: not buggy
4.7 nightly: buggy
4.8.1: buggy
4.8 nightly: buggy
master: buggy

Comment 11 Peter Hunt 2021-09-21 15:17:55 UTC
fixed by attached PR

Comment 15 Peter Hunt 2021-11-22 21:30:34 UTC
*** Bug 2025329 has been marked as a duplicate of this bug. ***

Comment 18 Peter Hunt 2021-11-29 20:46:49 UTC
PR merged

Comment 20 Sunil Choudhary 2021-12-09 10:14:13 UTC
Verified on 4.10.0-0.nightly-2021-12-06-201335.

Created pods and checked veth ports on a node while pods were running and after pods were deleted.

$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2021-12-06-201335   True        False         3h52m   Cluster version is 4.10.0-0.nightly-2021-12-06-201335

Comment 23 errata-xmlrpc 2022-03-10 16:09:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056

Comment 27 Red Hat Bugzilla 2023-09-15 01:14:56 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days


Note You need to log in before you can comment on or make changes to this bug.