Bug 2002805

Summary: Increase in RSS memory in CRI-O and Kubelet
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NodeAssignee: Peter Hunt <pehunt>
Node sub component: CRI-O QA Contact: MinLi <minmli>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: aos-bugs, kgordeev, minmli, nagrawal, rphillips, rsandu, rsevilla, schoudha, wking
Version: 4.8Keywords: Performance, TestBlocker
Target Milestone: ---   
Target Release: 4.8.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.8.10 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-09-21 08:03:13 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1986452    
Bug Blocks: 2002806    

Comment 1 Peter Hunt 2021-09-09 18:49:55 UTC
Introduced in 4.8.2, fully fixed in 4.8.10

Comment 4 MinLi 2021-09-14 06:46:06 UTC
I did a comparison between 4.8.11 and 4.9 nightly. Crio memory usage is in similar range.

$ oc get clusterversion 
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.11    True        False         96m     Cluster version is 4.8.11

$ oc get node 
NAME                                        STATUS   ROLES    AGE    VERSION
ip-10-0-50-127.us-east-2.compute.internal   Ready    master   126m   v1.21.1+9807387
ip-10-0-55-67.us-east-2.compute.internal    Ready    worker   117m   v1.21.1+9807387
ip-10-0-59-224.us-east-2.compute.internal   Ready    master   126m   v1.21.1+9807387
ip-10-0-69-239.us-east-2.compute.internal   Ready    worker   116m   v1.21.1+9807387
ip-10-0-79-82.us-east-2.compute.internal    Ready    master   125m   v1.21.1+9807387

$ oc debug node/ip-10-0-55-67.us-east-2.compute.internal
Starting pod/ip-10-0-55-67us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
...
sh-4.4# ps -ef | grep -e  "\/usr\/bin\/crio"
root        1319       1  2 04:35 ?        00:03:31 /usr/bin/crio   
sh-4.4# ps -p 1319 -o pid,rss,vsz,cmd
    PID   RSS    VSZ CMD
   1319 102772 2589876 /usr/bin/crio


$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.9.0-0.nightly-2021-09-01-193941   True        False         125m    Cluster version is 4.9.0-0.nightly-2021-09-01-193941

$ oc get nodes
NAME                                          STATUS   ROLES    AGE    VERSION
ip-10-0-134-113.ap-south-1.compute.internal   Ready    worker   157m   v1.22.0-rc.0+bbcc9ae
ip-10-0-136-105.ap-south-1.compute.internal   Ready    master   168m   v1.22.0-rc.0+bbcc9ae
ip-10-0-170-97.ap-south-1.compute.internal    Ready    master   168m   v1.22.0-rc.0+bbcc9ae
ip-10-0-183-52.ap-south-1.compute.internal    Ready    worker   157m   v1.22.0-rc.0+bbcc9ae
ip-10-0-193-186.ap-south-1.compute.internal   Ready    worker   156m   v1.22.0-rc.0+bbcc9ae
ip-10-0-201-41.ap-south-1.compute.internal    Ready    master   168m   v1.22.0-rc.0+bbcc9ae

$ oc debug node/ip-10-0-134-113.ap-south-1.compute.internal
Starting pod/ip-10-0-134-113ap-south-1computeinternal-debug ...
...

sh-4.4# ps -p 1295 -o pid,rss,vsz,cmd 
    PID   RSS    VSZ CMD
   1295 111392 2028600 /usr/bin/crio

Comment 7 errata-xmlrpc 2021-09-21 08:03:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.12 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3511