Description of problem: If the podnetworkconnectivitycheck crd is not available, the check-endpoints tool (currently found in the kube-apiserver and openshift-apiserver pods) should disable itself until the podnetworkconnectivitycheck crd is available.
From https://bugzilla.redhat.com/show_bug.cgi?id=1876166#c3, enabling PodNetworkConnectivityCheck in release builds didn't work as expected, the following message is continuously output in kube-apiserver-operator logs for a long time(more than 30 minutes),the PodNetworkConnectivityCheck crd still not was created, unable to continue verification, so assign bug back. I1010 10:30:01.198964 1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-apiserver-operator", Name:"kube-apiserver-operator", UID:"870c8cd3-5a47-4eb6-a9a7-2aed44f9f948", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'EndpointDetectionFailure' PodNetworkConnectivityCheck.controlplane.operator.openshift.io/kube-apiserver-ip-10-0-155-67.us-east-2.compute.internal-to-load-balancer-api-internal -n openshift-kube-apiserver: the server could not find the requested resource (post podnetworkconnectivitychecks.controlplane.operator.openshift.io)
This BZ is just for the check-endpoints tool, not the connectivitycheckcontrollers themselves. You can verify the check-endpoints containers are idle when the PodNetworkConnectivityCheck CRDdoes not exist (which is now the default).
Verified with OCP 4.6.0-0.nightly-2020-10-12-223649, Updated the KubeAPIServer as following, $ oc edit kubeapiserver cluster spec: unsupportedConfigOverrides: operator: enableConnectivityCheckController: "True" Wait for a while, check the kube-apiserver-operator log, the podnetworkconnectivitycheck crd is not available, I1013 04:15:43.234129 1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-apiserver-operator", Name:"kube-apiserver-operator", UID:"981f0ab2-418 a-4cdf-81b4-b2d0c1500f48", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'EndpointDetectionFailure' PodNetworkConnectivityCheck.controlplane.operato r.openshift.io/kube-apiserver-ip-xx-xx-xx-xx.us-east-2.compute.internal-to-etcd-server-ip-xx-x-xx-xxx.us-east-2.compute.internal -n openshift-kube-apiserver: the server could not find t he requested resource (post podnetworkconnectivitychecks.controlplane.operator.openshift.io) $ oc get pods -n openshift-apiserver NAME READY STATUS RESTARTS AGE apiserver-6c7864474f-5d7wp 1/2 CrashLoopBackOff 12 62m apiserver-6c7864474f-j9njx 1/2 CrashLoopBackOff 12 66m apiserver-6c7864474f-mflw7 1/2 CrashLoopBackOff 12 64m $ oc get pods -n openshift-kube-apiserver | grep kube-apiserver kube-apiserver-ip.xxx.us-east-2.compute.internal 4/5 CrashLoopBackOff 12 75m kube-apiserver-ip.xx.us-east-2.compute.internal 4/5 CrashLoopBackOff 12 73m kube-apiserver-ip.x.us-east-2.compute.internal 4/5 CrashLoopBackOff 12 69m Check if the openshift-apiserver-check-endpoints and kube-apiserver-check-endpoints containers is running on all masters, $ nodes=$(oc get no | grep master | awk '{print $1}') $ for no in $nodes; do oc debug node/$no -- chroot /host crictl ps | grep 'check-endpoints';done No running check-endpoints tools openshift-apiserver-check-endpoints and kube-apiserver-check-endpoints containers found, it is as expected.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196