Description of problem: ---------------------------- Upgrade ran on a disconnected cluster from version 4.5.7 to version 4.6 nightly without using force flag. I let the upgrade run for some time, but it seems like it stucked with this message: [kni@provisionhost-0-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.7 True True 3h49m Unable to apply 4.6.0-0.nightly-2020-08-26-032807: the update could not be applied [kni@provisionhost-0-0 ~]$ oc get co NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.6.0-0.nightly-2020-08-26-032807 False False True 19s cloud-credential 4.6.0-0.nightly-2020-08-26-032807 True False False 5h44m cluster-autoscaler 4.6.0-0.nightly-2020-08-26-032807 True False False 5h20m config-operator 4.6.0-0.nightly-2020-08-26-032807 True False False 5h21m console 4.6.0-0.nightly-2020-08-26-032807 True False True 3h26m csi-snapshot-controller 4.6.0-0.nightly-2020-08-26-032807 True False False 5h4m dns 4.5.7 True False False 5h27m etcd 4.6.0-0.nightly-2020-08-26-032807 True False False 5h27m image-registry 4.6.0-0.nightly-2020-08-26-032807 True False False 5h21m ingress 4.6.0-0.nightly-2020-08-26-032807 True False False 3h26m insights 4.6.0-0.nightly-2020-08-26-032807 True False False 5h25m kube-apiserver 4.6.0-0.nightly-2020-08-26-032807 True False False 5h26m kube-controller-manager 4.6.0-0.nightly-2020-08-26-032807 True False False 5h26m kube-scheduler 4.6.0-0.nightly-2020-08-26-032807 True False False 5h23m kube-storage-version-migrator 4.6.0-0.nightly-2020-08-26-032807 True False False 5h4m machine-api 4.6.0-0.nightly-2020-08-26-032807 True False False 5h17m machine-approver 4.6.0-0.nightly-2020-08-26-032807 True False False 5h26m machine-config 4.5.7 True False False 5h27m marketplace 4.6.0-0.nightly-2020-08-26-032807 True False False 3h26m monitoring 4.6.0-0.nightly-2020-08-26-032807 False False True 34m network 4.6.0-0.nightly-2020-08-26-032807 True False False 5h28m node-tuning 4.6.0-0.nightly-2020-08-26-032807 True False False 3h26m openshift-apiserver 4.6.0-0.nightly-2020-08-26-032807 False False False 13m openshift-controller-manager 4.6.0-0.nightly-2020-08-26-032807 True False False 5h23m openshift-samples 4.6.0-0.nightly-2020-08-26-032807 True False False 3h17m operator-lifecycle-manager 4.6.0-0.nightly-2020-08-26-032807 True False False 5h28m operator-lifecycle-manager-catalog 4.6.0-0.nightly-2020-08-26-032807 True False False 5h28m operator-lifecycle-manager-packageserver 4.6.0-0.nightly-2020-08-26-032807 False True False 51s service-ca 4.6.0-0.nightly-2020-08-26-032807 True False False 5h28m storage 4.6.0-0.nightly-2020-08-26-032807 True False False 3h27m Around 20 minutes after been stuck in this state ( for 100 minutes ), the cluster tried again to upgrade but arrive to the same state. [kni@provisionhost-0-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.7 True True 4h3m Working towards 4.6.0-0.nightly-2020-08-26-032807: 1% complete Operators condition: ------------------------ [kni@provisionhost-0-0 ~]$ oc get co NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.6.0-0.nightly-2020-08-26-032807 False False True 1s cloud-credential 4.6.0-0.nightly-2020-08-26-032807 True False False 5h59m cluster-autoscaler 4.6.0-0.nightly-2020-08-26-032807 True False False 5h34m config-operator 4.6.0-0.nightly-2020-08-26-032807 True False False 5h35m console 4.6.0-0.nightly-2020-08-26-032807 True False True 3h40m csi-snapshot-controller 4.6.0-0.nightly-2020-08-26-032807 True False False 5h18m dns 4.5.7 True False False 5h41m etcd 4.6.0-0.nightly-2020-08-26-032807 True False False 5h41m image-registry 4.6.0-0.nightly-2020-08-26-032807 True False False 5h35m ingress 4.6.0-0.nightly-2020-08-26-032807 True False False 3h40m insights 4.6.0-0.nightly-2020-08-26-032807 True False False 5h39m kube-apiserver 4.6.0-0.nightly-2020-08-26-032807 True False False 5h41m kube-controller-manager 4.6.0-0.nightly-2020-08-26-032807 True False False 5h41m kube-scheduler 4.6.0-0.nightly-2020-08-26-032807 True False False 5h38m kube-storage-version-migrator 4.6.0-0.nightly-2020-08-26-032807 True False False 5h18m machine-api 4.6.0-0.nightly-2020-08-26-032807 True False False 5h31m machine-approver 4.6.0-0.nightly-2020-08-26-032807 True False False 5h40m machine-config 4.5.7 True False False 5h41m marketplace 4.6.0-0.nightly-2020-08-26-032807 True False False 3h40m monitoring 4.6.0-0.nightly-2020-08-26-032807 False False True 48m network 4.6.0-0.nightly-2020-08-26-032807 True False False 5h42m node-tuning 4.6.0-0.nightly-2020-08-26-032807 True False False 3h40m openshift-apiserver 4.6.0-0.nightly-2020-08-26-032807 False False False 3m59s openshift-controller-manager 4.6.0-0.nightly-2020-08-26-032807 True False False 5h37m openshift-samples 4.6.0-0.nightly-2020-08-26-032807 True False False 3h32m operator-lifecycle-manager 4.6.0-0.nightly-2020-08-26-032807 True False False 5h42m operator-lifecycle-manager-catalog 4.6.0-0.nightly-2020-08-26-032807 True False False 5h42m operator-lifecycle-manager-packageserver 4.6.0-0.nightly-2020-08-26-032807 False True False 11m service-ca 4.6.0-0.nightly-2020-08-26-032807 True False False 5h42m storage 4.6.0-0.nightly-2020-08-26-032807 True False False 3h41m How reproducible: ---------------------- 2 out of 2 failed with the same steps, so i would say always. Steps to Reproduce: ---------------------- 1. Deploy disconnected cluster with IPV6 provisioning network and IPV4 baremetal network, with version 4.5.7 : https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/releasetag/4.5.7 2.Upgrade the cluster to version 4.6 nightly ( i used this one: https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/releasetag/4.6.0-0.nightly-2020-08-26-010422 or this one : https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/releasetag/4.6.0-0.nightly-2020-08-26-032807 ) without using force flag $ oc adm upgrade --to 4.6.0-0.nightly-2020-08-26-032807 Actual results: ---------------------- Cluster failed to upgrade to version 4.6 nightly after a long period of time. Expected results: ---------------------- Upgrade from 4.5.7 to 4.6 nightly passed successfully. Additional info: ---------------------- must- gather: could not add must-gather due to internal error in the cluster;
*** This bug has been marked as a duplicate of bug 1872742 ***