Description of problem: The alert was fired when upgrading build02 from 4.4.4 to 4.4.10. [FIRING:1] KubeDeploymentReplicasMismatch kube-state-metrics (etcd-quorum-guard https-main 10.131.2.16:8443 openshift-machine-config-operator kube-state-metrics-7749fc9947-xx4qj openshift-monitoring/k8s kube-state-metrics critical) Deployment openshift-machine-config-operator/etcd-quorum-guard has not matched the expected number of replicas for longer than 15 minutes. https://coreos.slack.com/archives/CHY2E1BL4/p1593117580479500 Must-Gather http://file.rdu.redhat.com/~hongkliu/dptp2020/bz1825000/ How reproducible: once Eventually upgrade was completed successfully. Is this expected during the upgrade? What should the cluster admin do after seeing this alert?
etcd-quorum-guard was born in the MCO repo because there was not yet an etcd operator. Now that we have an etcd operator, these components are maintained by that team, and will be moving over to the etcd operator repo at some point in the future.
Please re-test using the fixes applied to https://bugzilla.redhat.com/show_bug.cgi?id=1829923 and let's see if they helped. Otherwise we need a way to reproduce measure whether this is actually happening with any significant frequency.
I can't hit this issue in upgrade, @hongkliu, have u ever hit it more, pls let me know or reopen it again, thanks
As far I as know, CI clusters hit this only once.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196