Bug 1965283

Summary: 4.7->4.8 upgrades: cluster operators are not ready: openshift-controller-manager (Upgradeable=Unknown NoData: ), service-ca (Upgradeable=Unknown NoData:
Product: OpenShift Container Platform Reporter: Petr Muller <pmuller>
Component: openshift-controller-managerAssignee: Adam Kaplan <adam.kaplan>
openshift-controller-manager sub component: controller-manager QA Contact: wewang <wewang>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: aos-bugs
Version: 4.8   
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: controllers within openshift-controller-manager-operator issued conflicting updates when reconciling the state of openshift-controller-manager Consequence: openshift-controller-manager-operator did not record a proper status for the Upgradeable cluster operator condition Fix: refactored operator to use controllers that issued non-conflicting status updates Result: operator reports the Upgradeable status correctly
 Story Points: ---
Clone Of: Environment:
test: openshift-tests.[sig-arch][Early] Managed cluster should start all core operators [Skipped:Disconnected] [Suite:openshift/conformance/parallel]
Last Closed: 2021-07-27 23:10:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Petr Muller 2021-05-27 11:13:18 UTC 
Since May 26, the following test started to fail in the 4.7->4.8 upgrade jobs:

openshift-tests.[sig-arch][Early] Managed cluster should start all core operators [Skipped:Disconnected] [Suite:openshift/conformance/parallel]

fail [github.com/onsi/ginkgo.0-origin.0+incompatible/internal/leafnodes/runner.go:113]: May 27 06:11:45.984: Some cluster operators are not ready: openshift-controller-manager (Upgradeable=Unknown NoData: ), service-ca (Upgradeable=Unknown NoData: )

AWS Example job: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-aws-upgrade/1397836255068164096
GCP Example job: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.8-informing#periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-upgrade

AWS Testgrid: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.8-informing#periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-aws-upgrade
GCP Testgrid: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.8-informing#periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-upgrade
Comment 1 Petr Muller 2021-05-27 11:14:07 UTC 
GCP job link was wrong: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-upgrade/1397769933705711616
Comment 4 wewang 2021-06-08 02:17:09 UTC 
FROM testgrid, no ocm error now
AWS Testgrid: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.8-informing#periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-aws-upgrade
GCP Testgrid: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.8-informing#periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-upgrade
Comment 7 errata-xmlrpc 2021-07-27 23:10:12 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438