Description of problem: The kube-apiserver pod ran into CrashLoopBackOff due to the PodNetworkConnectivityCheck fail when the cluster upgrade from 4.6.0-rc.4 to 4.7 nightly build Version-Release number of selected component (if applicable): 4.6.0-rc.4 How reproducible: Steps to Reproduce: 1.install a cluster with version 4.6.0-rc.4 2.upgrade the cluster to 4.7.0-0.nightly-2020-10-17-034503 3. Actual results: 2.upgrade fail, co openshift-apiserver degrade, and namespace pods ran into CrashLoopBackOff status due to ConnectivityCheck fail Expected results: 2.upgrade succeed Additional info: $ oc get po -n openshift-kube-apiserver NAME READY STATUS RESTARTS AGE installer-5-ip-10-0-176-199.us-east-2.compute.internal 0/1 Completed 0 108m kube-apiserver-ip-10-0-147-197.us-east-2.compute.internal 4/5 CrashLoopBackOff 25 3h14m kube-apiserver-ip-10-0-176-199.us-east-2.compute.internal 4/5 CrashLoopBackOff 25 108m kube-apiserver-ip-10-0-214-165.us-east-2.compute.internal 4/5 CrashLoopBackOff 25 3h18m revision-pruner-4-ip-10-0-147-197.us-east-2.compute.internal 0/1 Completed 0 3h3m revision-pruner-4-ip-10-0-176-199.us-east-2.compute.internal 0/1 Completed 0 3h1m revision-pruner-4-ip-10-0-214-165.us-east-2.compute.internal 0/1 Completed 0 3h6m $ oc get pod kube-apiserver-ip-10-0-147-197.us-east-2.compute.internal -n openshift-kube-apiserver -o yaml ... - containerID: cri-o://05d81129a3fb841f1bcb564ac5a49300a05fd91624a71b3a5e35ef099d963902 image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4691dc29704c9cb06d2345894f1a8f074b58a0d208318c5218241388b0916e1b imageID: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4691dc29704c9cb06d2345894f1a8f074b58a0d208318c5218241388b0916e1b lastState: terminated: containerID: cri-o://05d81129a3fb841f1bcb564ac5a49300a05fd91624a71b3a5e35ef099d963902 exitCode: 0 finishedAt: "2020-10-20T11:09:50Z" reason: Completed startedAt: "2020-10-20T11:09:44Z" name: kube-apiserver-check-endpoints ready: false restartCount: 31 started: false state: waiting: message: back-off 5m0s restarting failed container=kube-apiserver-check-endpoints pod=kube-apiserver-ip-10-0-147-197.us-east-2.compute.internal_openshift-kube-apiserver(bb2901174f0fad53f457a459ecaaeb2e) reason: CrashLoopBackOff $ oc logs -n openshift-kube-apiserver kube-apiserver-ip-10-0-147-197.us-east-2.compute.internal -c kube-apiserver-check-endpoints ... E1020 10:43:43.154124 1 reflector.go:127] k8s.io/client-go.0/tools/cache/reflector.go:156: Failed to watch *v1alpha1.PodNetworkConnectivityCheck: failed to list *v1alpha1.PodNetworkConnectivityCheck: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I1020 10:43:43.345413 1 start_stop_controllers.go:70] The server doesn't have a resource type "podnetworkconnectivitychecks.controlplane.operator.openshift.io". ~
There is a similar bug in 4.7 :https://bugzilla.redhat.com/show_bug.cgi?id=1876166
Removing UpgradeBlocker from this older bug, to remove it from the suspect queue described in [1]. If you feel like this bug still needs to be a suspect, please add keyword again. [1]: https://github.com/openshift/enhancements/pull/475