Description of problem: KubeMacPool is logging cache of all allocated MAC addresses per each operation on a VM into logs. On a cluster with many VMs, this may mean millions of MACs logged every couple of days and exhaustion of available Node resources. Since KMP will be enabled by default, this issue is urgent. Version-Release number of selected component (if applicable): CNV 4.8 How reproducible: Always Steps to Reproduce: 1. Create multiple VMs 2. Check logs of the KubeMacPool Deploymetn Actual results: Each operation on a VM is accompanied by a dump of all allocated MACs. Expected results: Allocated MACs should be logged only in debug mode.
https://github.com/k8snetworkplumbingwg/kubemacpool/pull/310
Version-Release number of selected component: CNV: v.4.8.0 Server Version: v.4.8.0-fc.7 Kubernetes Version: v1.21.0-rc.0+4b2b6ff CNAO: v4.8.0-24 KMP: v4.8.0-19 Steps to reproduce: 1. create a namespace. 2. create 4 VMs (vm-1 to vm-4) using the attached files (kmp_log_vm_1.yaml, kmp_log_vm_2.yaml, kmp_log_vm_3.yaml, kmp_log_vm_4.yaml). 3. start all VMs (virtctl start vm-1 and so on) 4. make sure all VM's are running: $ oc get vmi 5. delete all VMs (oc delete command) 6. look at the logs produced: $ oc -n openshift-cnv get pods | grep mac $ oc -n openshift-cnv logs kubemacpool-mac-controller-manager-796bf7557c-4p4pm Repeat all steps in debug mode. How to set kubemacpool-mac-controller-manager deployment to debug mode: 1. shut down HCO (hyperconverged-cluster-operator) operator (change replicas of deployment to 0) $ oc scale deployment.apps/hco-operator -n openshift-cnv --replicas=0 2. shut down CNAO operator (change replicas of deployment to 0) Shut-down the cluster network operator: $ oc scale deployment.apps/cluster-network-addons-operator --replicas=0 -n openshift-cnv 3. change KMP deployment param to -v debug $ oc edit deployment -n openshift-cnv kubemacpool-mac-controller-manager (under: spec -> template -> spec -> containers -> to "v=debug")
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Virtualization 4.8.0 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2920