*** Bug 1816803 has been marked as a duplicate of this bug. ***
*** Bug 1822768 has been marked as a duplicate of this bug. ***
*** Bug 1840885 has been marked as a duplicate of this bug. ***
The dependent Bug 1813221 is not fixed, move it to Assigned
*** Bug 1853088 has been marked as a duplicate of this bug. ***
Tested with 4.3.0-0.nightly-2020-07-10-031338, and compared with 44 settings, see from https://bugzilla.redhat.com/show_bug.cgi?id=1813221#c41, there are a few problems 1. alertmanager/config-reloader container don't have resources.requests.cpu setting, and we should reduce the cpu request for alertmanager-proxy 2. should reduce resources.requests.cpu setting for grafana/grafana-proxy containers 3. should reduce resources.requests.cpu setting for kube-rbac-proxy-main/kube-rbac-proxy-self/kube-state-metrics containers 4. should reduce resources.requests.cpu setting for node-exporter/kube-rbac-proxy containers 5. should reduce resources.requests.cpu setting for openshift-state-metrics container 6. should reduce resources.requests.cpu setting for prometheus-adapter container 7. should reduce resources.requests.cpu setting for prometheus/thanos-sidecar/prometheus-proxy/prom-label-proxy containers, don't have resources.requests.cpu setting for prometheus-config-reloader/rules-configmap-reloader containers 8. should reduce resources.requests.cpu setting for prometheus-operator container 9. don't have resources.requests.cpu setting for telemeter-client/reload containers 10. should reduce resources.requests.cpu setting for thanos-querier/oauth-proxy containers # for i in $(kubectl -n openshift-monitoring get pod --no-headers | awk '{print $1}'); do echo $i; kubectl -n openshift-monitoring get pod $i -o go-template='{{range.spec.containers}}{{"Container Name: "}}{{.name}}{{"\r\nresources: "}}{{.resources}}{{"\n"}}{{end}}'; echo -e "\n"; done alertmanager-main-0 Container Name: alertmanager resources: map[requests:map[memory:200Mi]] Container Name: config-reloader resources: map[] Container Name: alertmanager-proxy resources: map[requests:map[cpu:10m memory:20Mi]] alertmanager-main-1 Container Name: alertmanager resources: map[requests:map[memory:200Mi]] Container Name: config-reloader resources: map[] Container Name: alertmanager-proxy resources: map[requests:map[cpu:10m memory:20Mi]] alertmanager-main-2 Container Name: alertmanager resources: map[requests:map[memory:200Mi]] Container Name: config-reloader resources: map[] Container Name: alertmanager-proxy resources: map[requests:map[cpu:10m memory:20Mi]] cluster-monitoring-operator-bcff5c786-b5c9d Container Name: cluster-monitoring-operator resources: map[requests:map[cpu:10m memory:50Mi]] grafana-65cc69bb4b-bf6sd Container Name: grafana resources: map[requests:map[cpu:100m memory:100Mi]] Container Name: grafana-proxy resources: map[requests:map[cpu:10m memory:20Mi]] kube-state-metrics-76f5c5b69-w5b5c Container Name: kube-rbac-proxy-main resources: map[requests:map[cpu:10m memory:40Mi]] Container Name: kube-rbac-proxy-self resources: map[requests:map[cpu:10m memory:40Mi]] Container Name: kube-state-metrics resources: map[requests:map[cpu:10m memory:40Mi]] node-exporter-42p2r Container Name: node-exporter resources: map[requests:map[cpu:102m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] node-exporter-6srcd Container Name: node-exporter resources: map[requests:map[cpu:102m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] node-exporter-78kd6 Container Name: node-exporter resources: map[requests:map[cpu:102m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] node-exporter-crpgl Container Name: node-exporter resources: map[requests:map[cpu:102m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] node-exporter-gdng4 Container Name: node-exporter resources: map[requests:map[cpu:102m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] node-exporter-psg2q Container Name: node-exporter resources: map[requests:map[cpu:102m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] openshift-state-metrics-d7f77b7c4-8lp5h Container Name: kube-rbac-proxy-main resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: kube-rbac-proxy-self resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: openshift-state-metrics resources: map[requests:map[cpu:100m memory:150Mi]] prometheus-adapter-6bc6f5fdbd-rnz72 Container Name: prometheus-adapter resources: map[requests:map[cpu:10m memory:20Mi]] prometheus-adapter-6bc6f5fdbd-rtv64 Container Name: prometheus-adapter resources: map[requests:map[cpu:10m memory:20Mi]] prometheus-k8s-0 Container Name: prometheus resources: map[requests:map[cpu:200m memory:1Gi]] Container Name: prometheus-config-reloader resources: map[] Container Name: rules-configmap-reloader resources: map[] Container Name: thanos-sidecar resources: map[requests:map[cpu:50m memory:100Mi]] Container Name: prometheus-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:10m memory:20Mi]] prometheus-k8s-1 Container Name: prometheus resources: map[requests:map[cpu:200m memory:1Gi]] Container Name: prometheus-config-reloader resources: map[] Container Name: rules-configmap-reloader resources: map[] Container Name: thanos-sidecar resources: map[requests:map[cpu:50m memory:100Mi]] Container Name: prometheus-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:10m memory:20Mi]] prometheus-operator-5cbf6b9ff4-6zlbl Container Name: prometheus-operator resources: map[requests:map[cpu:10m memory:60Mi]] telemeter-client-7b5b466948-8m67j Container Name: telemeter-client resources: map[] Container Name: reload resources: map[] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] thanos-querier-788d89f746-kn44n Container Name: thanos-querier resources: map[requests:map[cpu:10m memory:12Mi]] Container Name: oauth-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:10m memory:20Mi]] thanos-querier-788d89f746-sjkr8 Container Name: thanos-querier resources: map[requests:map[cpu:10m memory:12Mi]] Container Name: oauth-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:10m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:10m memory:20Mi]]
Tested with 4.3.0-0.nightly-2020-07-25-094223, issue is fixed # for i in $(kubectl -n openshift-monitoring get pod --no-headers | awk '{print $1}'); do echo $i; kubectl -n openshift-monitoring get pod $i -o go-template='{{range.spec.containers}}{{"Container Name: "}}{{.name}}{{"\r\nresources: "}}{{.resources}}{{"\n"}}{{end}}'; echo -e "\n"; done alertmanager-main-0 Container Name: alertmanager resources: map[requests:map[cpu:4m memory:200Mi]] Container Name: config-reloader resources: map[requests:map[cpu:1m]] Container Name: alertmanager-proxy resources: map[requests:map[cpu:1m memory:20Mi]] alertmanager-main-1 Container Name: alertmanager resources: map[requests:map[cpu:4m memory:200Mi]] Container Name: config-reloader resources: map[requests:map[cpu:1m]] Container Name: alertmanager-proxy resources: map[requests:map[cpu:1m memory:20Mi]] alertmanager-main-2 Container Name: alertmanager resources: map[requests:map[cpu:4m memory:200Mi]] Container Name: config-reloader resources: map[requests:map[cpu:1m]] Container Name: alertmanager-proxy resources: map[requests:map[cpu:1m memory:20Mi]] cluster-monitoring-operator-7c46c4dd7c-t9x9c Container Name: cluster-monitoring-operator resources: map[requests:map[cpu:10m memory:50Mi]] grafana-bbdfb8846-xr44n Container Name: grafana resources: map[requests:map[cpu:4m memory:100Mi]] Container Name: grafana-proxy resources: map[requests:map[cpu:1m memory:20Mi]] kube-state-metrics-6b78dc8f78-gctsz Container Name: kube-rbac-proxy-main resources: map[requests:map[cpu:1m memory:40Mi]] Container Name: kube-rbac-proxy-self resources: map[requests:map[cpu:1m memory:40Mi]] Container Name: kube-state-metrics resources: map[requests:map[cpu:2m memory:40Mi]] node-exporter-9dj5w Container Name: node-exporter resources: map[requests:map[cpu:8m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:30Mi]] node-exporter-mnlwf Container Name: node-exporter resources: map[requests:map[cpu:8m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:30Mi]] node-exporter-nt4nz Container Name: node-exporter resources: map[requests:map[cpu:8m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:30Mi]] node-exporter-t6jsj Container Name: node-exporter resources: map[requests:map[cpu:8m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:30Mi]] node-exporter-ww4lf Container Name: node-exporter resources: map[requests:map[cpu:8m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:30Mi]] node-exporter-zf7ch Container Name: node-exporter resources: map[requests:map[cpu:8m memory:180Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:30Mi]] openshift-state-metrics-cc7d4d797-mqs6p Container Name: kube-rbac-proxy-main resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: kube-rbac-proxy-self resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: openshift-state-metrics resources: map[requests:map[cpu:1m memory:150Mi]] prometheus-adapter-756f54898b-hbpq9 Container Name: prometheus-adapter resources: map[requests:map[cpu:1m memory:20Mi]] prometheus-adapter-756f54898b-n5rng Container Name: prometheus-adapter resources: map[requests:map[cpu:1m memory:20Mi]] prometheus-k8s-0 Container Name: prometheus resources: map[requests:map[cpu:70m memory:1Gi]] Container Name: prometheus-config-reloader resources: map[requests:map[cpu:1m]] Container Name: rules-configmap-reloader resources: map[requests:map[cpu:1m]] Container Name: thanos-sidecar resources: map[requests:map[cpu:1m memory:100Mi]] Container Name: prometheus-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:1m memory:20Mi]] prometheus-k8s-1 Container Name: prometheus resources: map[requests:map[cpu:70m memory:1Gi]] Container Name: prometheus-config-reloader resources: map[requests:map[cpu:1m]] Container Name: rules-configmap-reloader resources: map[requests:map[cpu:1m]] Container Name: thanos-sidecar resources: map[requests:map[cpu:1m memory:100Mi]] Container Name: prometheus-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:1m memory:20Mi]] prometheus-operator-7f6b94c5c8-5l2sq Container Name: prometheus-operator resources: map[requests:map[cpu:5m memory:60Mi]] telemeter-client-59455b549d-mcvbb Container Name: telemeter-client resources: map[requests:map[cpu:1m]] Container Name: reload resources: map[requests:map[cpu:1m]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:20Mi]] thanos-querier-6d6b7957f7-fc6mz Container Name: thanos-querier resources: map[requests:map[cpu:5m memory:12Mi]] Container Name: oauth-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:1m memory:20Mi]] thanos-querier-6d6b7957f7-npkgb Container Name: thanos-querier resources: map[requests:map[cpu:5m memory:12Mi]] Container Name: oauth-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: kube-rbac-proxy resources: map[requests:map[cpu:1m memory:20Mi]] Container Name: prom-label-proxy resources: map[requests:map[cpu:1m memory:20Mi]]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.3.31 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3180