Description of problem: In large OpenShift 4 environments we are seeing around 83'000 messages "Throttling request took 1.183079627s, request: GET: ..." in cluster-monitoring-operator logs on daily basis. This is related to the amount of objects and CRD's in the OpenShift 4 - Cluster. As increasing `config.Burst` in `oc` client seems to address this, we are requesting the same in the cluster-monitoring operator (probably cfg.Burst in cluster-monitoring-operator). Version-Release number of selected component (if applicable): - 4.5 and 4.6 How reproducible: - Always (depending on the numbers of CRD's in the OpenShift - Cluster Steps to Reproduce: 1. Add many CRD's to OpenShift 4 and add workloads 2. Watch logs from cluster-monitoring-operator Actual results: The `cluster-monitoring-operator` is constantly reporting `Throttling request took 1.183079627s, request: GET: ...` and thus throttling requests on client side. Expected results: No `Throttling request took 1.183079627s, request: GET: ...` being reported even when we have lots of CRD's and workload on a cluster. Additional info:
Thanks for the bug report, the solution makes sense. Lowered priority to medium from high, as from what I understand this does not actually have an effect on any component? It just logs a lot, that is all? Feel free to correct me!
(In reply to Lili Cosic from comment #3) > Thanks for the bug report, the solution makes sense. Lowered priority to > medium from high, as from what I understand this does not actually have an > effect on any component? It just logs a lot, that is all? Feel free to > correct me! That is fine. But please make sure you fix it in timely manner or at least put in documentation/explanation to help people understand that it does not harm the environment and is only informational (client side throttling).
Lowering the severity of this bug to low as it doesn't impact the use of any component of the monitoring stack.
tested with 4.7.0-0.nightly-2020-12-08-141245, added many CRD's to OpenShift 4 and add workloads, no `Throttling request took **` info in cluster-monitoring-operator, and there's error with `oc get pod -A'
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633