Description of problem: When I run cluster-density with 500 namespaces on a cluster with 20 nodes, sharp peaks are observed multiple (~10) times which show that the ovn-dbchecker container consumes ~500MiB RSS. Version-Release number of selected component (if applicable): 4.8.0-0.nightly-2021-06-13-101614 How reproducible: Always Steps to Reproduce: 1. Run cluster-density on 20 nodes with 500 namespaces Actual results: Sharp peaks for ovn-dbchecker Expected results: No sharp peaks Additional info:
root cause is ovsdb-tool loading the entire database and consuming a lot of CPU to search for the server id. https://bugzilla.redhat.com/show_bug.cgi?id=2011468 is filed to track this on the OVSDB side
removing the dependent ovsdb-tool bug. We will migrate to using ovn-appctl in ovn-kubernetes: https://github.com/ovn-org/ovn-kubernetes/pull/2554
Verified on 4.10.0-0.nightly-2021-12-02-140313. 20 node cluster and 500 iteration cluster-density. No spikes in ovn-dbchecker memory usage - it was stable at 22-26 MiB throughout the run.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056