Bug 1742753
| Summary: | Upgrade from 4.1 to 4.2 fails due to cloud-credential-operator | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Clayton Coleman <ccoleman> |
| Component: | Cloud Credential Operator | Assignee: | Joel Diaz <jdiaz> |
| Status: | CLOSED ERRATA | QA Contact: | Oleg Nesterov <olnester> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 4.2.0 | CC: | jdiaz, sdodson, wking |
| Target Milestone: | --- | ||
| Target Release: | 4.2.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-10-16 06:36:19 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Clayton Coleman
2019-08-16 20:13:23 UTC
It appears that some CredentialsRequest objects are set to Ignored==True (b/c they are for a different cloud/platform), but a previous ProvisionFailure==True is also set and causing cloud-cred-operator to be degraded. PR https://github.com/openshift/cloud-credential-operator/pull/103 addresses this case by clearing other conditions when deciding to ignore a CredentialsRequest. Will need to backport this to 4.2 once merged. Verified. Cluster operator cloud-credential is upgraded successfully. [onest@localhost bugzilla]$ oc get clusteroperator NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.1.11 True False False 7m57s cloud-credential 4.1.11 True False False 20m cluster-autoscaler 4.1.11 True False False 20m console 4.1.11 True False False 11m dns 4.1.11 True False False 19m image-registry 4.1.11 True False False 14m ingress 4.1.11 True False False 14m kube-apiserver 4.1.11 True False False 17m kube-controller-manager 4.1.11 True False False 19m kube-scheduler 4.1.11 True False False 17m machine-api 4.1.11 True False False 20m machine-config 4.1.11 True False False 19m marketplace 4.1.11 True False False 14m monitoring 4.1.11 True False False 13m network 4.1.11 True False False 19m node-tuning 4.1.11 True False False 16m openshift-apiserver 4.1.11 True False False 16m openshift-controller-manager 4.1.11 True False False 19m openshift-samples 4.1.11 True False False 14m operator-lifecycle-manager 4.1.11 True False False 19m operator-lifecycle-manager-catalog 4.1.11 True False False 19m service-ca 4.1.11 True False False 20m service-catalog-apiserver 4.1.11 True False False 16m service-catalog-controller-manager 4.1.11 True False False 16m storage 4.1.11 True False False 15m [onest@localhost bugzilla]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.1.11 True False 7m49s Cluster version is 4.1.11 [onest@localhost bugzilla]$ oc adm upgrade --to-image registry.svc.ci.openshift.org/ocp/release:4.1.0-0.nightly-2019-08-14-005846 Updating to release image registry.svc.ci.openshift.org/ocp/release:4.1.0-0.nightly-2019-08-14-005846 [onest@localhost bugzilla]$ oc get clusteroperator NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.1.0-0.nightly-2019-08-14-005846 True False False 48m cloud-credential 4.1.0-0.nightly-2019-08-14-005846 True False False 60m cluster-autoscaler 4.1.0-0.nightly-2019-08-14-005846 True False False 61m console 4.1.0-0.nightly-2019-08-14-005846 True False False 51m dns 4.1.0-0.nightly-2019-08-14-005846 True False False 60m image-registry 4.1.0-0.nightly-2019-08-14-005846 True False False 12m ingress 4.1.0-0.nightly-2019-08-14-005846 True False False 54m kube-apiserver 4.1.0-0.nightly-2019-08-14-005846 True False False 58m kube-controller-manager 4.1.0-0.nightly-2019-08-14-005846 True False False 59m kube-scheduler 4.1.0-0.nightly-2019-08-14-005846 True False False 57m machine-api 4.1.0-0.nightly-2019-08-14-005846 True False False 60m machine-config 4.1.0-0.nightly-2019-08-14-005846 True False False 60m marketplace 4.1.0-0.nightly-2019-08-14-005846 True False False 9m46s monitoring 4.1.0-0.nightly-2019-08-14-005846 True False False 12m network 4.1.0-0.nightly-2019-08-14-005846 True False False 60m node-tuning 4.1.0-0.nightly-2019-08-14-005846 True False False 30m openshift-apiserver 4.1.0-0.nightly-2019-08-14-005846 True False False 12m openshift-controller-manager 4.1.0-0.nightly-2019-08-14-005846 True False False 60m openshift-samples 4.1.0-0.nightly-2019-08-14-005846 True False False 31m operator-lifecycle-manager 4.1.0-0.nightly-2019-08-14-005846 True False False 59m operator-lifecycle-manager-catalog 4.1.0-0.nightly-2019-08-14-005846 True False False 59m service-ca 4.1.0-0.nightly-2019-08-14-005846 True False False 60m service-catalog-apiserver 4.1.0-0.nightly-2019-08-14-005846 True False False 57m service-catalog-controller-manager 4.1.0-0.nightly-2019-08-14-005846 True False False 57m storage 4.1.0-0.nightly-2019-08-14-005846 True False False 31m [onest@localhost bugzilla]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.1.0-0.nightly-2019-08-14-005846 True False 7m46s Cluster version is 4.1.0-0.nightly-2019-08-14-005846 oh, sorry It is wrong results. I will provide results for upgrade to 4.2 a bit later The issue with cloud credential operator upgrade is fixed Upgrade operation is still working because of dns, machine-config, marketplace and network operator are not upgraded [onest@localhost bugzilla]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.1.11 True False 42m Cluster version is 4.1.11 [onest@localhost bugzilla]$ oc get clusteroperator NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.1.11 True False False 42m cloud-credential 4.1.11 True False False 53m cluster-autoscaler 4.1.11 True False False 53m console 4.1.11 True False False 45m dns 4.1.11 True False False 53m image-registry 4.1.11 True False False 47m ingress 4.1.11 True False False 47m kube-apiserver 4.1.11 True False False 52m kube-controller-manager 4.1.11 True False False 51m kube-scheduler 4.1.11 True False False 51m machine-api 4.1.11 True False False 53m machine-config 4.1.11 True False False 52m marketplace 4.1.11 True False False 47m monitoring 4.1.11 True False False 46m network 4.1.11 True False False 53m node-tuning 4.1.11 True False False 42m openshift-apiserver 4.1.11 True False False 50m openshift-controller-manager 4.1.11 True False False 52m openshift-samples 4.1.11 True False False 48m operator-lifecycle-manager 4.1.11 True False False 52m operator-lifecycle-manager-catalog 4.1.11 True False False 52m service-ca 4.1.11 True False False 53m service-catalog-apiserver 4.1.11 True False False 50m service-catalog-controller-manager 4.1.11 True False False 50m storage 4.1.11 True False False 48m [onest@localhost bugzilla]$ oc get clusteroperator NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.2.0-0.ci-2019-08-20-015949 True False False 5h20m cloud-credential 4.2.0-0.ci-2019-08-20-015949 True False False 5h31m cluster-autoscaler 4.2.0-0.ci-2019-08-20-015949 True False False 5h31m console 4.2.0-0.ci-2019-08-20-015949 True False False 4h25m dns 4.1.11 True False False 5h31m image-registry 4.2.0-0.ci-2019-08-20-015949 True False False 5h25m ingress 4.2.0-0.ci-2019-08-20-015949 True False False 5h25m insights 4.2.0-0.ci-2019-08-20-015949 True False False 4h30m kube-apiserver 4.2.0-0.ci-2019-08-20-015949 True False False 5h30m kube-controller-manager 4.2.0-0.ci-2019-08-20-015949 True False False 5h29m kube-scheduler 4.2.0-0.ci-2019-08-20-015949 True False False 5h29m machine-api 4.2.0-0.ci-2019-08-20-015949 True False False 5h31m machine-config 4.1.11 True False False 5h31m marketplace 4.1.11 True False False 5h26m monitoring 4.2.0-0.ci-2019-08-20-015949 True False False 5h24m network 4.1.11 True False False 5h31m node-tuning 4.2.0-0.ci-2019-08-20-015949 True False False 4h29m openshift-apiserver 4.2.0-0.ci-2019-08-20-015949 True False False 5h28m openshift-controller-manager 4.2.0-0.ci-2019-08-20-015949 True False False 5h31m openshift-samples 4.2.0-0.ci-2019-08-20-015949 True False False 4h20m operator-lifecycle-manager 4.2.0-0.ci-2019-08-20-015949 True False False 5h30m operator-lifecycle-manager-catalog 4.2.0-0.ci-2019-08-20-015949 True False False 5h30m operator-lifecycle-manager-packageserver 4.2.0-0.ci-2019-08-20-015949 True False False 4h29m service-ca 4.2.0-0.ci-2019-08-20-015949 True False False 5h31m service-catalog-apiserver 4.2.0-0.ci-2019-08-20-015949 True False False 5h28m service-catalog-controller-manager 4.2.0-0.ci-2019-08-20-015949 True False False 5h28m storage 4.2.0-0.ci-2019-08-20-015949 True False False 4h30m [onest@localhost bugzilla]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.1.11 True True 4h38m Working towards registry.svc.ci.openshift.org/ocp/release:4.2.0-0.ci-2019-08-20-015949: downloading update Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922 |