Description of problem: Prometheus statefulset is reconciled on pod deletion but not on scale down. Version-Release number of selected component (if applicable): multiple. I've tested this on 4 clusters with the following versions: 4.1.7 3.11.117 3.11.98 3.11.69 How reproducible: always. Steps to Reproduce: 1. oc scale statefulset.apps/prometheus-k8s --replicas=1 2. The second sts replica is not created again. Yet the prometheus-k8s-1 pod is deleted, a new one is created instead. Actual results: the second sts replica is not created again when the sts is scaled down. Expected results: the statefulset to be reconciled both on pod deletion of statefulset scale down. Additional info: - Nothing suspicious in the prom operator logs - Prometheus operator is expecting 2 replicas, but only one sts is up: $ oc -n openshift-monitoring get prometheus k8s -o yaml | grep -i replicas replicas: 2 $ oc -n openshift-monitoring describe sts prometheus-k8s | grep -i replicas Replicas: 1 desired | 1 total
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062