Bug 1973423
| Summary: | Several operators degraded because Failed to create pod sandbox when installing an sts cluster | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Douglas Smith <dosmith> | |
| Component: | Networking | Assignee: | Douglas Smith <dosmith> | |
| Networking sub component: | multus | QA Contact: | Weibin Liang <weliang> | |
| Status: | CLOSED ERRATA | Docs Contact: | ||
| Severity: | urgent | |||
| Priority: | urgent | CC: | lwan, nagrawal, slaznick, surbania, weliang, wking, yunjiang, zzhao | |
| Version: | 4.9 | Keywords: | Regression, TestBlocker | |
| Target Milestone: | --- | |||
| Target Release: | 4.9.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 1972167 | |||
| : | 1973690 (view as bug list) | Environment: | ||
| Last Closed: | 2021-10-18 17:35:14 UTC | Type: | --- | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1972167 | |||
@zzhao this issue has gone on 4.9.0-0.nightly-2021-06-23-160041 and 4.8.0-0.nightly-2021-06-23-232238, it causeb by a apiserver-auth bug [1] , and has fixed on both 4.8 and 4.9. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1974716 (In reply to wang lin from comment #3) > @zzhao this issue has gone on 4.9.0-0.nightly-2021-06-23-160041 > and 4.8.0-0.nightly-2021-06-23-232238, it causeb by a apiserver-auth bug [1] > , and has fixed on both 4.8 and 4.9. > > > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1974716 Thanks, wang lin. Move this bug to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759 |
Hi, Checked this on 4.9.0-0.nightly-2021-06-21-191858 pods can be created and cluster are working well. However, see below logs, there are some error like 'E0622 03:04:50.985314 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized', is this related for this issue? $ oc logs network-metrics-daemon-9txvn -n openshift-multus error: a container name must be specified for pod network-metrics-daemon-9txvn, choose one of: [network-metrics-daemon kube-rbac-proxy] [zzhao@dhcp-0-105 ~]$ oc logs network-metrics-daemon-9txvn -n openshift-multus -c network-metrics-daemon I0622 02:34:57.888541 1 main.go:45] Version:35937a4dff1f348be51f38d0236cc657c67f3d6e I0622 02:34:57.888629 1 main.go:46] Starting with config{ :9091 ip-10-0-208-125.us-east-2.compute.internal} W0622 02:34:57.888707 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I0622 02:34:57.890466 1 controller.go:42] Setting up event handlers I0622 02:34:57.890618 1 podmetrics.go:101] Serving network metrics I0622 02:34:57.890637 1 controller.go:101] Starting pod controller I0622 02:34:57.890642 1 controller.go:104] Waiting for informer caches to sync I0622 02:34:57.991265 1 controller.go:109] Starting workers I0622 02:34:57.991332 1 controller.go:114] Started workers I0622 02:34:57.991792 1 controller.go:192] Received pod 'cloud-credential-operator-6d76bd65c9-wrqlv' I0622 02:34:57.991891 1 controller.go:151] Successfully synced 'openshift-cloud-credential-operator/cloud-credential-operator-6d76bd65c9-wrqlv' I0622 02:34:57.991907 1 controller.go:192] Received pod 'pod-identity-webhook-846d9d6864-wjnp2' I0622 02:34:57.991933 1 controller.go:151] Successfully synced 'openshift-cloud-credential-operator/pod-identity-webhook-846d9d6864-wjnp2' I0622 02:34:57.991947 1 controller.go:192] Received pod 'cluster-storage-operator-5ccccf59fb-sz5n5' I0622 02:34:57.992066 1 controller.go:192] Received pod 'csi-snapshot-controller-799d75d6ff-tp7hs' I0622 02:34:57.992268 1 controller.go:151] Successfully synced 'openshift-cluster-storage-operator/cluster-storage-operator-5ccccf59fb-sz5n5' I0622 02:34:57.992289 1 controller.go:192] Received pod 'openshift-controller-manager-operator-6b5599c78b-k56gq' I0622 02:34:57.992335 1 controller.go:151] Successfully synced 'openshift-controller-manager-operator/openshift-controller-manager-operator-6b5599c78b-k56gq' I0622 02:34:57.992351 1 controller.go:192] Received pod 'controller-manager-xzqf5' I0622 02:34:57.992380 1 controller.go:151] Successfully synced 'openshift-controller-manager/controller-manager-xzqf5' I0622 02:34:57.992412 1 controller.go:192] Received pod 'dns-default-rtnhj' I0622 02:34:57.992447 1 controller.go:151] Successfully synced 'openshift-dns/dns-default-rtnhj' I0622 02:34:57.992479 1 controller.go:192] Received pod 'ingress-operator-7df55b5b5d-7n2wp' I0622 02:34:57.992518 1 controller.go:151] Successfully synced 'openshift-ingress-operator/ingress-operator-7df55b5b5d-7n2wp' I0622 02:34:57.992532 1 controller.go:192] Received pod 'insights-operator-5dbb5795c4-sm5st' I0622 02:34:57.992555 1 controller.go:151] Successfully synced 'openshift-insights/insights-operator-5dbb5795c4-sm5st' I0622 02:34:57.992563 1 controller.go:192] Received pod 'installer-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:34:57.992596 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/installer-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:34:57.992611 1 controller.go:192] Received pod 'installer-4-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:34:57.992629 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/installer-4-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:34:57.992636 1 controller.go:192] Received pod 'kube-storage-version-migrator-operator-869b7c9756-2rtw4' I0622 02:34:57.992659 1 controller.go:151] Successfully synced 'openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-869b7c9756-2rtw4' I0622 02:34:57.992667 1 controller.go:192] Received pod 'cluster-autoscaler-operator-f866c6b45-rd58n' I0622 02:34:57.992688 1 controller.go:151] Successfully synced 'openshift-machine-api/cluster-autoscaler-operator-f866c6b45-rd58n' I0622 02:34:57.992695 1 controller.go:192] Received pod 'machine-api-controllers-87db98d98-g25rq' I0622 02:34:57.992718 1 controller.go:151] Successfully synced 'openshift-machine-api/machine-api-controllers-87db98d98-g25rq' I0622 02:34:57.992724 1 controller.go:192] Received pod 'machine-api-operator-6fd86796d7-vqc96' I0622 02:34:57.992755 1 controller.go:151] Successfully synced 'openshift-machine-api/machine-api-operator-6fd86796d7-vqc96' I0622 02:34:57.992763 1 controller.go:192] Received pod 'machine-config-controller-54669c494c-5fpkw' I0622 02:34:57.992786 1 controller.go:151] Successfully synced 'openshift-machine-config-operator/machine-config-controller-54669c494c-5fpkw' I0622 02:34:57.992793 1 controller.go:192] Received pod 'marketplace-operator-57b6444fb4-vqsw8' I0622 02:34:57.992814 1 controller.go:151] Successfully synced 'openshift-marketplace/marketplace-operator-57b6444fb4-vqsw8' I0622 02:34:57.992821 1 controller.go:192] Received pod 'multus-admission-controller-q6jh6' I0622 02:34:57.992845 1 controller.go:151] Successfully synced 'openshift-multus/multus-admission-controller-q6jh6' I0622 02:34:57.992852 1 controller.go:192] Received pod 'network-metrics-daemon-9txvn' I0622 02:34:57.992874 1 controller.go:151] Successfully synced 'openshift-multus/network-metrics-daemon-9txvn' I0622 02:34:57.992880 1 controller.go:192] Received pod 'network-check-target-bzbnz' I0622 02:34:57.992902 1 controller.go:151] Successfully synced 'openshift-network-diagnostics/network-check-target-bzbnz' I0622 02:34:57.992909 1 controller.go:192] Received pod 'service-ca-86cc56464b-q9ncg' I0622 02:34:57.992930 1 controller.go:151] Successfully synced 'openshift-service-ca/service-ca-86cc56464b-q9ncg' I0622 02:34:57.992944 1 controller.go:151] Successfully synced 'openshift-cluster-storage-operator/csi-snapshot-controller-799d75d6ff-tp7hs' I0622 02:35:01.497458 1 controller.go:192] Received pod 'installer-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:01.497514 1 controller.go:151] Successfully synced 'openshift-etcd/installer-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:23.106032 1 controller.go:192] Received pod 'revision-pruner-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:23.106109 1 controller.go:151] Successfully synced 'openshift-etcd/revision-pruner-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:25.281310 1 controller.go:192] Received pod 'apiserver-c9687d7b8-ps7w5' I0622 02:35:25.281472 1 controller.go:151] Successfully synced 'openshift-oauth-apiserver/apiserver-c9687d7b8-ps7w5' I0622 02:35:25.749115 1 controller.go:192] Received pod 'apiserver-f648fb4cc-z8k4p' I0622 02:35:25.749217 1 controller.go:151] Successfully synced 'openshift-apiserver/apiserver-f648fb4cc-z8k4p' I0622 02:35:34.564011 1 controller.go:192] Received pod 'revision-pruner-4-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:34.564149 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/revision-pruner-4-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:55.924665 1 controller.go:192] Received pod 'installer-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:35:55.924782 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/installer-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 02:36:04.310180 1 controller.go:151] Successfully synced 'openshift-controller-manager/controller-manager-xzqf5' E0622 02:36:43.850685 1 reflector.go:380] github.com/openshift/network-metrics-daemon/main.go:82: Failed to watch *v1.Pod: the server has asked for the client to provide credentials (get pods) E0622 02:36:43.936223 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:36:46.239531 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:36:50.799784 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:37:01.598995 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:37:15.241534 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:37:44.850946 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:38:17.762338 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:38:56.792524 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:39:42.251292 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:40:36.662517 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:41:13.091817 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:41:54.514284 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:42:34.057946 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:43:18.126961 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:43:56.619617 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:44:35.414980 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:45:25.789611 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:46:02.348112 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:46:38.445968 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:47:19.274516 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:48:06.396826 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:49:02.273564 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:49:41.068456 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:50:19.983005 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:51:12.562388 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:51:48.761815 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:52:44.725596 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:53:35.629297 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:54:21.346136 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:54:52.197851 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:55:26.949822 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:56:15.169814 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:57:14.429087 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:57:46.814948 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:58:34.661821 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:59:06.439280 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 02:59:57.202159 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:00:36.250154 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:01:11.449795 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:01:57.685036 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:02:44.011820 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:03:22.368998 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:04:05.065772 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:04:50.985314 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:05:28.593856 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:06:07.058441 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:07:00.718967 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:07:41.574703 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:08:37.992897 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:09:16.908177 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:10:13.740854 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:10:46.666085 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:11:45.975731 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:12:18.206472 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:12:54.876508 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:13:45.311416 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:14:22.558983 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:15:01.906453 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:15:59.894021 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:16:52.151353 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:17:46.184877 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:18:38.094178 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:19:13.584262 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:19:56.436937 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized E0622 03:20:53.348965 1 reflector.go:178] github.com/openshift/network-metrics-daemon/main.go:82: Failed to list *v1.Pod: Unauthorized I0622 03:21:47.604348 1 controller.go:192] Received pod 'controller-manager-qsw77' I0622 03:21:47.604514 1 controller.go:151] Successfully synced 'openshift-controller-manager/controller-manager-qsw77' I0622 03:25:39.578779 1 controller.go:192] Received pod 'revision-pruner-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:25:39.580078 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/revision-pruner-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:25:56.270691 1 controller.go:151] Successfully synced 'openshift-controller-manager/controller-manager-qsw77' I0622 03:25:58.799467 1 controller.go:192] Received pod 'controller-manager-mknsn' I0622 03:25:58.799521 1 controller.go:151] Successfully synced 'openshift-controller-manager/controller-manager-mknsn' I0622 03:26:43.636768 1 controller.go:192] Received pod 'installer-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:26:43.637230 1 controller.go:151] Successfully synced 'openshift-etcd/installer-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:27:50.701601 1 controller.go:151] Successfully synced 'openshift-etcd/etcd-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:27:56.663070 1 controller.go:151] Successfully synced 'openshift-apiserver/apiserver-f648fb4cc-z8k4p' I0622 03:28:00.739384 1 controller.go:192] Received pod 'apiserver-7ffbcb5b4-869pw' I0622 03:28:00.739620 1 controller.go:151] Successfully synced 'openshift-apiserver/apiserver-7ffbcb5b4-869pw' I0622 03:33:51.350397 1 controller.go:192] Received pod 'installer-5-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:33:51.350478 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/installer-5-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:03.150383 1 controller.go:192] Received pod 'console-645cdb66cf-twgcn' I0622 03:34:03.150568 1 controller.go:151] Successfully synced 'openshift-console/console-645cdb66cf-twgcn' I0622 03:34:03.774209 1 controller.go:192] Received pod 'revision-pruner-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:03.774303 1 controller.go:151] Successfully synced 'openshift-etcd/revision-pruner-3-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:05.293316 1 controller.go:192] Received pod 'openshift-kube-scheduler-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:05.293450 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/openshift-kube-scheduler-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:05.776633 1 controller.go:192] Received pod 'oauth-openshift-6969dc978d-2x675' I0622 03:34:05.776687 1 controller.go:151] Successfully synced 'openshift-authentication/oauth-openshift-6969dc978d-2x675' I0622 03:34:09.740283 1 controller.go:192] Received pod 'installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:09.740417 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:27.263935 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/openshift-kube-scheduler-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:33.077476 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/installer-2-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:39.764536 1 controller.go:192] Received pod 'installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:39.764618 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:41.214436 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:56.253018 1 controller.go:151] Successfully synced 'openshift-authentication/oauth-openshift-6969dc978d-2x675' I0622 03:34:56.922479 1 controller.go:192] Received pod 'installer-8-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:34:56.922671 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/installer-8-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:35:03.321483 1 controller.go:192] Received pod 'oauth-openshift-554f8d56fb-4m5tt' I0622 03:35:03.321556 1 controller.go:151] Successfully synced 'openshift-authentication/oauth-openshift-554f8d56fb-4m5tt' I0622 03:35:21.115922 1 controller.go:192] Received pod 'revision-pruner-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:35:21.116069 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/revision-pruner-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:35:55.981510 1 controller.go:192] Received pod 'installer-5-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:35:55.981588 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/installer-5-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:36:11.263150 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/kube-controller-manager-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:36:36.707280 1 controller.go:192] Received pod 'revision-pruner-5-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:36:36.707422 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/revision-pruner-5-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:37:47.590118 1 controller.go:192] Received pod 'installer-9-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:37:47.590828 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/installer-9-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:39:01.835148 1 controller.go:192] Received pod 'installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:39:01.835338 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/installer-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:39:17.261536 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/kube-controller-manager-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:39:41.259729 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/kube-apiserver-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:04.816153 1 controller.go:192] Received pod 'installer-7-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:04.816276 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/installer-7-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:18.291934 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/openshift-kube-scheduler-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:41.498663 1 controller.go:192] Received pod 'revision-pruner-9-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:41.498713 1 controller.go:151] Successfully synced 'openshift-kube-apiserver/revision-pruner-9-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:43.615561 1 controller.go:192] Received pod 'revision-pruner-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:40:43.615642 1 controller.go:151] Successfully synced 'openshift-kube-controller-manager/revision-pruner-6-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:41:14.537529 1 controller.go:192] Received pod 'revision-pruner-7-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:41:14.537580 1 controller.go:151] Successfully synced 'openshift-kube-scheduler/revision-pruner-7-ip-10-0-208-125.us-east-2.compute.internal' I0622 03:41:39.478589 1 controller.go:192] Received pod 'console-644f7477d8-2q6g4' I0622 03:41:39.478643 1 controller.go:151] Successfully synced 'openshift-console/console-644f7477d8-2q6g4' I0622 03:43:06.255956 1 controller.go:151] Successfully synced 'openshift-console/console-645cdb66cf-twgcn'