Hide Forgot
Description of problem: The journal is being spammed with this message about 850 times per second: Jan 06 18:42:09 ip-172-31-54-162.ec2.internal atomic-openshift-master-api[43301]: E0106 18:42:09.730919 43301 cache.go:332] error synchronizing: clusterrole.rbac.authorization.k8s.io "cluster-capacity-role" not found [root@starter-us-east-1-master-25064 ~]# journalctl --no-pager --since="3 minutes ago" |grep '18:42:09' |grep -c clusterrole.rbac.authorization.k8s.io 850 [root@starter-us-east-1-master-25064 ~]# journalctl --no-pager --since="3 minutes ago" |grep '18:42:10' |grep -c clusterrole.rbac.authorization.k8s.io 843 [root@starter-us-east-1-master-25064 ~]# journalctl --no-pager --since="5 minutes ago" |grep '18:42:11' |grep -c clusterrole.rbac.authorization.k8s.io 850 Version-Release number of selected component (if applicable): oc v3.7.9 How reproducible: It's happening on all 3 masters in starter-us-east-1. I haven't seen it elsewhere. Steps to Reproduce: 1. journalctl -fl 2. 3. Actual results: Many instances of this message scroll by every second: Jan 06 18:48:16 ip-172-31-55-199.ec2.internal atomic-openshift-master-api[16734]: E0106 18:48:16.652886 16734 cache.go:332] error synchronizing: clusterrole.rbac.authorization.k8s.io "cluster-capacity-role" not found Expected results: Additional info:
There was a role binding, but no role. I have deleted the invalid role binding from the cluster in question to work around this BZ. We should log, maybe even regularly. But not 850 times per second.
Eric I am sympatethic to the request, but how do you propose we handle this? Should we write a new logging library that can deal with this ?
If the message needs to be logged, rate limit it yourself (write a helper so others can use it if it doesn't exist). If we don't really need to know about this message and/or if there is nothing the admin can or should do to correct the situtation, we should reduce the log level.
WIP upstream PR https://github.com/kubernetes/kubernetes/pull/58307
Hello, Do we have any update on this issue. Thanks, Kedar Salunkhe
This has been fixed with https://github.com/openshift/origin/pull/21522
Verified. I have check the kube api log, no such log report now. oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.0.0-0.nightly-2019-03-06-074438 True False 43m Cluster version is 4.0.0-0.nightly-2019-03-06-074438
*** Bug 1699937 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758