Description of problem: Cluster fails to install when using OVNKubernetes network type. Version-Release number of selected component (if applicable): unreleased-master-1504-g40f7fa5ed322f06d37d7dfd1e8c222499b3be718-dirty Tested several nightlies, that had marked as passed, but none could be installed. How reproducible: Very Steps to Reproduce: 1. Modify install-config to use OVN Kubernetes 2. Run standard install 3. Actual results: Glenns-MacBook-Pro-2:gw.lo gwest$ oc get clusteroperators NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE network False True False 29m Glenns-MacBook-Pro-2:gw.lo gwest$ apiVersion: v1 items: - apiVersion: config.openshift.io/v1 kind: ClusterVersion metadata: creationTimestamp: "2019-08-02T16:47:37Z" generation: 1 name: version resourceVersion: "6106" selfLink: /apis/config.openshift.io/v1/clusterversions/version uid: 3c163626-b545-11e9-a400-0050561f2525 spec: channel: stable-4.2 clusterID: f0a9b4ea-486e-4260-b265-e1ec5d766462 upstream: https://api.openshift.com/api/upgrades_info/v1/graph status: availableUpdates: null conditions: - lastTransitionTime: "2019-08-02T16:48:06Z" status: "False" type: Available - lastTransitionTime: "2019-08-02T17:15:14Z" message: |- Multiple errors are preventing progress: * Could not update oauthclient "console" (257 of 431): the server does not recognize this resource, check extension API servers * Could not update rolebinding "openshift/cluster-samples-operator-openshift-edit" (207 of 431): resource may have been deleted * Could not update servicemonitor "openshift-apiserver-operator/openshift-apiserver-operator" (426 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-authentication-operator/authentication-operator" (394 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-cluster-version/cluster-version-operator" (6 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-controller-manager-operator/openshift-controller-manager-operator" (430 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-image-registry/image-registry" (400 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-kube-apiserver-operator/kube-apiserver-operator" (410 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-kube-controller-manager-operator/kube-controller-manager-operator" (414 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-kube-scheduler-operator/kube-scheduler-operator" (418 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-machine-api/cluster-autoscaler-operator" (145 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-operator-lifecycle-manager/olm-operator" (420 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-service-catalog-apiserver-operator/openshift-service-catalog-apiserver-operator" (403 of 431): the server does not recognize this resource, check extension API servers * Could not update servicemonitor "openshift-service-catalog-controller-manager-operator/openshift-service-catalog-controller-manager-operator" (406 of 431): the server does not recognize this resource, check extension API servers reason: MultipleErrors status: "True" type: Failing - lastTransitionTime: "2019-08-02T16:48:06Z" message: 'Unable to apply 4.2.0-0.ci-2019-08-02-141506: an unknown error has occurred' reason: MultipleErrors status: "True" type: Progressing - lastTransitionTime: "2019-08-02T16:48:06Z" message: 'Unable to retrieve available updates: currently installed version 4.2.0-0.ci-2019-08-02-141506 not found in the "stable-4.2" channel' reason: RemoteFailed status: "False" type: RetrievedUpdates desired: force: false image: registry.svc.ci.openshift.org/ocp/release@sha256:477e024e6199c0f22db049c7d01c6ce02e454cc5b6f6d06b3a1a5c6dbc138d5f version: 4.2.0-0.ci-2019-08-02-141506 history: - completionTime: null image: registry.svc.ci.openshift.org/ocp/release@sha256:477e024e6199c0f22db049c7d01c6ce02e454cc5b6f6d06b3a1a5c6dbc138d5f startedTime: "2019-08-02T16:48:06Z" state: Partial verified: false version: 4.2.0-0.ci-2019-08-02-141506 observedGeneration: 1 versionHash: lGT54qUj3u4= kind: List metadata: resourceVersion: "" selfLink: "" Expected results: Working Cluster. Additional info:
The installer may not succeed (danw is working on some random bugs), but you should be left with a reasonably functional cluster for development work.
Last ci build that marked green, just tested, again, it fails. openshift-install unreleased-master-1506-g885a44213514c81507c62af9d7271e176baa5c05-dirty built from commit 885a44213514c81507c62af9d7271e176baa5c05 release image registry.svc.ci.openshift.org/ocp/release@sha256:56d57b177645e7ffdabf776ad321fe14053c17cf80020338a7f5df5702dd24ec Every 5.0s: oc get clusteroperators Glenns-MacBook-Pro-2.local: Mon Aug 5 10:43:00 2019 NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE network False True False 10m Network never will become ready.
Glenns-MacBook-Pro-2:gw.lo gwest$ ./clusteroperatorstatus.sh apiVersion: v1 items: - apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: "2019-08-05T15:32:55Z" generation: 1 name: network resourceVersion: "2308" selfLink: /apis/config.openshift.io/v1/clusteroperators/network uid: 4bc3a12b-b796-11e9-a820-0050561f2525 spec: {} status: conditions: - lastTransitionTime: "2019-08-05T15:32:56Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T15:32:56Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T15:32:56Z" message: DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T15:32:56Z" message: The network is starting up reason: Startup status: "False" type: Available
DEBUG OpenShift Installer v4.2.0-201908010219-dirty DEBUG Built from commit 1f8da8a771253e74db3bde6758acac2fdbfac0d3 Glenns-MacBook-Pro-2:gw.lo gwest$ ./clusteroperatorstatus.sh apiVersion: v1 items: - apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: "2019-08-05T16:23:58Z" generation: 1 name: network resourceVersion: "2276" selfLink: /apis/config.openshift.io/v1/clusteroperators/network uid: 6d9839d0-b79d-11e9-9c10-0050561f2525 spec: {} status: conditions: - lastTransitionTime: "2019-08-05T16:23:59Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T16:23:59Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T16:24:00Z" message: DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T16:24:00Z" message: The network is starting up reason: Startup status: "False" type: Available extension: null relatedObjects: - group: ""
Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE openshift-apiserver-operator pod/openshift-apiserver-operator-5c64784d47-mkq8l 1/1 Running 1 3m19s openshift-cloud-credential-operator pod/cloud-credential-operator-75fcf9d7f6-lwr7r 0/1 Error 1 3m30s openshift-cluster-machine-approver pod/machine-approver-5fb9454f5b-j6tj4 1/1 Running 0 3m32s openshift-cluster-version pod/cluster-version-operator-79996b9db4-mv7tc 1/1 Running 0 3m13s openshift-controller-manager-operator pod/openshift-controller-manager-operator-76cf7864d9-4z2sv 1/1 Running 1 3m19s openshift-dns-operator pod/dns-operator-67ffdc648b-58jt8 0/1 ContainerCreating 0 6s openshift-dns-operator pod/dns-operator-b5b485b4d-tjj4t 1/1 Running 2 3m30s openshift-etcd pod/etcd-member-control-plane-0 2/2 Running 0 2m27s openshift-etcd pod/etcd-member-control-plane-1 2/2 Running 0 2m22s openshift-etcd pod/etcd-member-control-plane-2 2/2 Running 0 3m37s openshift-insights pod/insights-operator-7b6c67955-4vwc7 0/1 Terminating 1 3m25s openshift-kube-apiserver-operator pod/kube-apiserver-operator-65c654d984-sw6pw 1/1 Running 1 3m19s openshift-kube-controller-manager-operator pod/kube-controller-manager-operator-8685fddd78-fmvl2 1/1 Running 1 3m20s openshift-kube-scheduler-operator pod/openshift-kube-scheduler-operator-5994687fb-7jmnf 1/1 Running 1 3m19s openshift-machine-api pod/machine-api-operator-766c9b669c-qw2r5 1/1 Running 0 2m30s openshift-machine-config-operator pod/machine-config-operator-57b8d4db67-rbbb9 1/1 Running 0 3m31s openshift-multus pod/multus-5c2cq 1/1 Running 0 3m25s openshift-multus pod/multus-6w4j4 1/1 Running 0 3m25s openshift-multus pod/multus-6xhnn 1/1 Running 0 3m25s openshift-multus pod/multus-admission-controller-bb57p 0/1 ContainerCreating 0 3m21s openshift-multus pod/multus-admission-controller-j9cn9 0/1 ContainerCreating 0 3m23s openshift-multus pod/multus-admission-controller-nxq7m 0/1 ContainerCreating 0 3m23s openshift-multus pod/multus-bm2z9 1/1 Running 0 3m25s openshift-multus pod/multus-j2c5b 1/1 Running 0 3m21s openshift-multus pod/multus-jsnwv 1/1 Running 0 3m23s openshift-network-operator pod/network-operator-6f5fb67c68-thqrb 1/1 Running 0 3m32s openshift-operator-lifecycle-manager pod/catalog-operator-6dd75c7d87-btfxz 0/1 ContainerCreating 0 3m25s openshift-operator-lifecycle-manager pod/olm-operator-854cffcff7-r7vfk 0/1 ContainerCreating 0 3m25s
Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE openshift-apiserver-operator pod/openshift-apiserver-operator-5c64784d47-mkq8l 1/1 Running 1 3m19s openshift-cloud-credential-operator pod/cloud-credential-operator-75fcf9d7f6-lwr7r 0/1 Error 1 3m30s openshift-cluster-machine-approver pod/machine-approver-5fb9454f5b-j6tj4 1/1 Running 0 3m32s openshift-cluster-version pod/cluster-version-operator-79996b9db4-mv7tc 1/1 Running 0 3m13s openshift-controller-manager-operator pod/openshift-controller-manager-operator-76cf7864d9-4z2sv 1/1 Running 1 3m19s openshift-dns-operator pod/dns-operator-67ffdc648b-58jt8 0/1 ContainerCreating 0 6s openshift-dns-operator pod/dns-operator-b5b485b4d-tjj4t 1/1 Running 2 3m30s openshift-etcd pod/etcd-member-control-plane-0 2/2 Running 0 2m27s openshift-etcd pod/etcd-member-control-plane-1 2/2 Running 0 2m22s openshift-etcd pod/etcd-member-control-plane-2 2/2 Running 0 3m37s openshift-insights pod/insights-operator-7b6c67955-4vwc7 0/1 Terminating 1 3m25s openshift-kube-apiserver-operator pod/kube-apiserver-operator-65c654d984-sw6pw 1/1 Running 1 3m19s openshift-kube-controller-manager-operator pod/kube-controller-manager-operator-8685fddd78-fmvl2 1/1 Running 1 3m20s openshift-kube-scheduler-operator pod/openshift-kube-scheduler-operator-5994687fb-7jmnf 1/1 Running 1 3m19s openshift-machine-api pod/machine-api-operator-766c9b669c-qw2r5 1/1 Running 0 2m30s openshift-machine-config-operator pod/machine-config-operator-57b8d4db67-rbbb9 1/1 Running 0 3m31s openshift-multus pod/multus-5c2cq 1/1 Running 0 3m25s openshift-multus pod/multus-6w4j4 1/1 Running 0 3m25s openshift-multus pod/multus-6xhnn 1/1 Running 0 3m25s openshift-multus pod/multus-admission-controller-bb57p 0/1 ContainerCreating 0 3m21s openshift-multus pod/multus-admission-controller-j9cn9 0/1 ContainerCreating 0 3m23s openshift-multus pod/multus-admission-controller-nxq7m 0/1 ContainerCreating 0 3m23s openshift-multus pod/multus-bm2z9 1/1 Running 0 3m25s openshift-multus pod/multus-j2c5b 1/1 Running 0 3m21s openshift-multus pod/multus-jsnwv 1/1 Running 0 3m23s openshift-network-operator pod/network-operator-6f5fb67c68-thqrb 1/1 Running 0 3m32s openshift-operator-lifecycle-manager pod/catalog-operator-6dd75c7d87-btfxz 0/1 ContainerCreating 0 3m25s openshift-operator-lifecycle-manager pod/olm-operator-854cffcff7-r7vfk 0/1 ContainerCreating 0 3m25s openshift-ovn-kubernetes pod/ovnkube-master-7fb7c59fd6-7v6t8 4/4 Running 0 3m13s openshift-ovn-kubernetes pod/ovnkube-node-cdt97 3/3 Running 0 3m12s openshift-ovn-kubernetes pod/ovnkube-node-fbm2d 3/3 Running 0 3m11s openshift-ovn-kubernetes pod/ovnkube-node-rx55q 3/3 Running 0 3m11s openshift-ovn-kubernetes pod/ovnkube-node-sf2mh 3/3 Running 0 3m12s openshift-ovn-kubernetes pod/ovnkube-node-swmn2 3/3 Running 1 3m11s openshift-ovn-kubernetes pod/ovnkube-node-tj92z 3/3 Running 0 3m12s openshift-service-ca-operator pod/service-ca-operator-667d77b8b9-c24fp 1/1 Running 1 3m21s openshift-vertical-pod-autoscaler pod/vertical-pod-autoscaler-operator-5fc74c8579-nv6gv 1/1 Running 2 3m26s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 172.30.0.1 <none> 443/TCP 4m16s openshift-apiserver-operator service/metrics ClusterIP 172.30.103.121 <none> 443/TCP 3m21s openshift-authentication-operator service/metrics ClusterIP 172.30.201.244 <none> 443/TCP 3m23s openshift-cloud-credential-operator service/controller-manager-service ClusterIP 172.30.174.49 <none> 443/TCP 3m31s openshift-cluster-version service/cluster-version-operator ClusterIP 172.30.142.233 <none> 9099/TCP 4m20s openshift-controller-manager-operator service/metrics ClusterIP 172.30.24.116 <none> 443/TCP 3m22s openshift-etcd service/etcd ClusterIP 172.30.140.110 <none> 2379/TCP,9979/TCP 4m17s openshift-etcd service/host-etcd ClusterIP None <none> 2379/TCP 4m8s openshift-kube-apiserver-operator service/metrics ClusterIP 172.30.88.35 <none> 443/TCP 3m23s openshift-kube-controller-manager-operator service/metrics ClusterIP 172.30.151.164 <none> 443/TCP 3m23s openshift-kube-scheduler-operator service/metrics ClusterIP 172.30.19.0 <none> 443/TCP 3m23s openshift-machine-api service/cluster-autoscaler-operator ClusterIP 172.30.37.67 <none> 443/TCP,8080/TCP 3m27s openshift-machine-api service/machine-api-operator ClusterIP 172.30.124.45 <none> 8080/TCP 2m34s openshift-multus service/multus-admission-controller ClusterIP 172.30.254.133 <none> 443/TCP 3m25s openshift-operator-lifecycle-manager service/catalog-operator-metrics ClusterIP 172.30.61.49 <none> 8081/TCP 3m30s openshift-operator-lifecycle-manager service/olm-operator-metrics ClusterIP 172.30.176.100 <none> 8081/TCP 3m31s openshift-ovn-kubernetes service/ovnkube-db ClusterIP None <none> 6641/TCP,6642/TCP 3m14s openshift-service-catalog-apiserver-operator service/metrics ClusterIP 172.30.147.187 <none> 443/TCP 3m20s openshift-service-catalog-controller-manager-operator service/metrics ClusterIP 172.30.5.48 <none> 443/TCP 3m22s NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE openshift-multus daemonset.apps/multus 6 6 6 6 6 kubernetes.io/os=linux 3m25s openshift-multus daemonset.apps/multus-admission-controller 3 3 0 3 0 node-role.kubernetes.io/master= 3m23s openshift-ovn-kubernetes daemonset.apps/ovnkube-node 6 6 6 6 6 beta.kubernetes.io/os=linux 3m13s NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE openshift-apiserver-operator deployment.apps/openshift-apiserver-operator 1/1 1 1 3m20s openshift-authentication-operator deployment.apps/authentication-operator 0/1 0 0 2m52s openshift-cloud-credential-operator deployment.apps/cloud-credential-operator 0/1 1 0 3m30s openshift-cluster-machine-approver deployment.apps/machine-approver 1/1 1 1 3m32s openshift-cluster-node-tuning-operator deployment.apps/cluster-node-tuning-operator 0/1 0 0 3m31s openshift-cluster-storage-operator deployment.apps/cluster-storage-operator 0/1 0 0 3m31s openshift-cluster-version deployment.apps/cluster-version-operator 1/1 1 1 4m13s openshift-controller-manager-operator deployment.apps/openshift-controller-manager-operator 1/1 1 1 3m19s openshift-dns-operator deployment.apps/dns-operator 1/1 1 1 3m30s openshift-image-registry deployment.apps/cluster-image-registry-operator 0/1 0 0 2m49s openshift-ingress-operator deployment.apps/ingress-operator 0/1 0 0 2m49s openshift-insights deployment.apps/insights-operator 0/1 0 0 3m28s openshift-kube-apiserver-operator deployment.apps/kube-apiserver-operator 1/1 1 1 3m20s openshift-kube-controller-manager-operator deployment.apps/kube-controller-manager-operator 1/1 1 1 3m21s openshift-kube-scheduler-operator deployment.apps/openshift-kube-scheduler-operator 1/1 1 1 3m20s openshift-machine-api deployment.apps/machine-api-operator 1/1 1 1 2m30s openshift-machine-config-operator deployment.apps/machine-config-operator 1/1 1 1 3m31s openshift-marketplace deployment.apps/marketplace-operator 0/1 0 0 3m29s openshift-monitoring deployment.apps/cluster-monitoring-operator 0/1 0 0 3m23s openshift-network-operator deployment.apps/network-operator 1/1 1 1 3m32s openshift-operator-lifecycle-manager deployment.apps/catalog-operator 0/1 1 0 3m25s openshift-operator-lifecycle-manager deployment.apps/olm-operator 0/1 1 0 3m25s openshift-ovn-kubernetes deployment.apps/ovnkube-master 1/1 1 1 3m14s openshift-service-ca-operator deployment.apps/service-ca-operator 1/1 1 1 3m22s openshift-service-catalog-apiserver-operator deployment.apps/openshift-service-catalog-apiserver-operator 0/1 0 0 3m19s openshift-service-catalog-controller-manager-operator deployment.apps/openshift-service-catalog-controller-manager-operator 0/1 0 0 3m20s openshift-vertical-pod-autoscaler deployment.apps/vertical-pod-autoscaler-operator 1/1 1 1 3m26s NAMESPACE NAME DESIRED CURRENT READY AGE openshift-apiserver-operator replicaset.apps/openshift-apiserver-operator-5c64784d47 1 1 1 3m20s openshift-authentication-operator replicaset.apps/authentication-operator-6fc4b64446 1 0 0 2m52s openshift-cloud-credential-operator replicaset.apps/cloud-credential-operator-75fcf9d7f6 1 1 0 3m30s openshift-cluster-machine-approver replicaset.apps/machine-approver-5fb9454f5b 1 1 1 3m32s openshift-cluster-node-tuning-operator replicaset.apps/cluster-node-tuning-operator-6d64dbb9fc 1 0 0 3m31s openshift-cluster-storage-operator replicaset.apps/cluster-storage-operator-75d8554649 1 0 0 3m31s openshift-cluster-version replicaset.apps/cluster-version-operator-79996b9db4 1 1 1 3m13s openshift-cluster-version replicaset.apps/cluster-version-operator-7bd9f465c4 0 0 0 4m13s openshift-controller-manager-operator replicaset.apps/openshift-controller-manager-operator-76cf7864d9 1 1 1 3m19s openshift-dns-operator replicaset.apps/dns-operator-67ffdc648b 1 1 0 6s openshift-dns-operator replicaset.apps/dns-operator-b5b485b4d 1 1 1 3m30s openshift-image-registry replicaset.apps/cluster-image-registry-operator-784574746d 1 0 0 2m49s openshift-ingress-operator replicaset.apps/ingress-operator-7f79b89c57 1 0 0 2m49s openshift-insights replicaset.apps/insights-operator-64c548fbfb 0 0 0 3m27s openshift-insights replicaset.apps/insights-operator-7b6c67955 0 0 0 3m25s openshift-kube-apiserver-operator replicaset.apps/kube-apiserver-operator-65c654d984 1 1 1 3m20s openshift-kube-controller-manager-operator replicaset.apps/kube-controller-manager-operator-8685fddd78 1 1 1 3m20s openshift-kube-scheduler-operator replicaset.apps/openshift-kube-scheduler-operator-5994687fb 1 1 1 3m20s openshift-machine-api replicaset.apps/machine-api-operator-766c9b669c 1 1 1 2m30s openshift-machine-config-operator replicaset.apps/machine-config-operator-57b8d4db67 1 1 1 3m31s openshift-marketplace replicaset.apps/marketplace-operator-98bb96567 1 0 0 3m29s openshift-monitoring replicaset.apps/cluster-monitoring-operator-dd65594f9 1 0 0 3m23s openshift-network-operator replicaset.apps/network-operator-6f5fb67c68 1 1 1 3m32s openshift-operator-lifecycle-manager replicaset.apps/catalog-operator-6dd75c7d87 1 1 0 3m25s openshift-operator-lifecycle-manager replicaset.apps/olm-operator-854cffcff7 1 1 0 3m25s openshift-ovn-kubernetes replicaset.apps/ovnkube-master-7fb7c59fd6 1 1 1 3m13s openshift-service-ca-operator replicaset.apps/service-ca-operator-667d77b8b9 1 1 1 3m22s openshift-service-catalog-apiserver-operator replicaset.apps/openshift-service-catalog-apiserver-operator-96f649554 1 0 0 3m19s openshift-service-catalog-controller-manager-operator replicaset.apps/openshift-service-catalog-controller-manager-operator-79c97695c4 1 0 0 3m20s openshift-vertical-pod-autoscaler replicaset.apps/vertical-pod-autoscaler-operator-5fc74c8579 1 1 1 3m26s Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE openshift-apiserver-operator pod/openshift-apiserver-operator-5c64784d47-mkq8l 0/1 CrashLoopBackOff 3 7m35s openshift-cloud-credential-operator pod/cloud-credential-operator-75fcf9d7f6-lwr7r 1/1 Running 5 7m46s openshift-cluster-machine-approver pod/machine-approver-5fb9454f5b-j6tj4 1/1 Running 0 7m48s openshift-cluster-version pod/cluster-version-operator-79996b9db4-mv7tc 1/1 Running 0 7m29s openshift-controller-manager-operator pod/openshift-controller-manager-operator-76cf7864d9-4z2sv 0/1 CrashLoopBackOff 3 7m35s openshift-dns-operator pod/dns-operator-67ffdc648b-58jt8 0/1 Error 4 4m22s openshift-etcd pod/etcd-member-control-plane-0 2/2 Running 0 6m43s openshift-etcd pod/etcd-member-control-plane-1 2/2 Running 0 6m38s openshift-etcd pod/etcd-member-control-plane-2 2/2 Running 0 7m53s openshift-insights pod/insights-operator-64c548fbfb-ppd9j 1/1 Running 0 26s openshift-kube-apiserver-operator pod/kube-apiserver-operator-65c654d984-sw6pw 0/1 CrashLoopBackOff 3 7m35s openshift-kube-controller-manager-operator pod/kube-controller-manager-operator-8685fddd78-fmvl2 0/1 CrashLoopBackOff 3 7m36s openshift-kube-scheduler-operator pod/openshift-kube-scheduler-operator-5994687fb-7jmnf 0/1 CrashLoopBackOff 3 7m35s openshift-machine-api pod/machine-api-operator-766c9b669c-qw2r5 1/1 Running 0 6m46s openshift-machine-config-operator pod/machine-config-operator-57b8d4db67-rbbb9 1/1 Running 0 7m47s openshift-multus pod/multus-5c2cq 1/1 Running 0 7m41s openshift-multus pod/multus-6w4j4 1/1 Running 0 7m41s openshift-multus pod/multus-6xhnn 1/1 Running 0 7m41s openshift-multus pod/multus-admission-controller-bb57p 0/1 ContainerCreating 0 7m37s openshift-multus pod/multus-admission-controller-j9cn9 0/1 ContainerCreating 0 7m39s openshift-multus pod/multus-admission-controller-nxq7m 0/1 ContainerCreating 0 7m39s openshift-multus pod/multus-bm2z9 1/1 Running 0 7m41s openshift-multus pod/multus-j2c5b 1/1 Running 0 7m37s openshift-multus pod/multus-jsnwv 1/1 Running 0 7m39s openshift-network-operator pod/network-operator-6f5fb67c68-thqrb 1/1 Running 0 7m48s openshift-operator-lifecycle-manager pod/catalog-operator-6dd75c7d87-btfxz 0/1 ContainerCreating 0 7m41s openshift-operator-lifecycle-manager pod/olm-operator-854cffcff7-r7vfk 0/1 ContainerCreating 0 7m41s openshift-ovn-kubernetes pod/ovnkube-master-7fb7c59fd6-7v6t8 4/4 Running 0 7m29s openshift-ovn-kubernetes pod/ovnkube-node-cdt97 3/3 Running 0 7m28s openshift-ovn-kubernetes pod/ovnkube-node-fbm2d 3/3 Running 0 7m27s openshift-ovn-kubernetes pod/ovnkube-node-rx55q 3/3 Running 0 7m27s openshift-ovn-kubernetes pod/ovnkube-node-sf2mh 3/3 Running 0 7m28s openshift-ovn-kubernetes pod/ovnkube-node-swmn2 3/3 Running 1 7m27s openshift-ovn-kubernetes pod/ovnkube-node-tj92z 3/3 Running 0 7m28s openshift-service-ca-operator pod/service-ca-operator-667d77b8b9-c24fp 1/1 Running 4 7m37s openshift-vertical-pod-autoscaler pod/vertical-pod-autoscaler-operator-5fc74c8579-nv6gv 1/1 Running 5 7m42s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 172.30.0.1 <none> 443/TCP 8m32s openshift-apiserver-operator service/metrics ClusterIP 172.30.103.121 <none> 443/TCP 7m37s openshift-authentication-operator service/metrics ClusterIP 172.30.201.244 <none> 443/TCP 7m39s openshift-cloud-credential-operator service/controller-manager-service ClusterIP 172.30.174.49 <none> 443/TCP 7m47s openshift-cluster-version service/cluster-version-operator ClusterIP 172.30.142.233 <none> 9099/TCP 8m36s openshift-controller-manager-operator service/metrics ClusterIP 172.30.24.116 <none> 443/TCP 7m38s openshift-etcd service/etcd ClusterIP 172.30.140.110 <none> 2379/TCP,9979/TCP 8m33s openshift-etcd service/host-etcd ClusterIP None <none> 2379/TCP 8m24s openshift-kube-apiserver-operator service/metrics ClusterIP 172.30.88.35 <none> 443/TCP 7m39s openshift-kube-controller-manager-operator service/metrics ClusterIP 172.30.151.164 <none> 443/TCP 7m39s openshift-kube-scheduler-operator service/metrics ClusterIP 172.30.19.0 <none> 443/TCP 7m39s openshift-machine-api service/cluster-autoscaler-operator ClusterIP 172.30.37.67 <none> 443/TCP,8080/TCP 7m43s openshift-machine-api service/machine-api-operator ClusterIP 172.30.124.45 <none> 8080/TCP 6m50s openshift-multus service/multus-admission-controller ClusterIP 172.30.254.133 <none> 443/TCP 7m41s openshift-operator-lifecycle-manager service/catalog-operator-metrics ClusterIP 172.30.61.49 <none> 8081/TCP 7m46s openshift-operator-lifecycle-manager service/olm-operator-metrics ClusterIP 172.30.176.100 <none> 8081/TCP 7m47s openshift-ovn-kubernetes service/ovnkube-db ClusterIP None <none> 6641/TCP,6642/TCP 7m30s openshift-service-catalog-apiserver-operator service/metrics ClusterIP 172.30.147.187 <none> 443/TCP 7m36s openshift-service-catalog-controller-manager-operator service/metrics ClusterIP 172.30.5.48 <none> 443/TCP 7m38s NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE openshift-multus daemonset.apps/multus 6 6 6 6 6 kubernetes.io/os=linux 7m41s openshift-multus daemonset.apps/multus-admission-controller 3 3 0 3 0 node-role.kubernetes.io/master= 7m39s openshift-ovn-kubernetes daemonset.apps/ovnkube-node 6 6 6 6 6 beta.kubernetes.io/os=linux 7m29s NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE openshift-apiserver-operator deployment.apps/openshift-apiserver-operator 0/1 1 0 7m36s openshift-authentication-operator deployment.apps/authentication-operator 0/1 0 0 7m8s openshift-cloud-credential-operator deployment.apps/cloud-credential-operator 1/1 1 1 7m46s openshift-cluster-machine-approver deployment.apps/machine-approver 1/1 1 1 7m48s openshift-cluster-node-tuning-operator deployment.apps/cluster-node-tuning-operator 0/1 0 0 7m47s openshift-cluster-storage-operator deployment.apps/cluster-storage-operator 0/1 0 0 7m47s openshift-cluster-version deployment.apps/cluster-version-operator 1/1 1 1 8m29s openshift-controller-manager-operator deployment.apps/openshift-controller-manager-operator 0/1 1 0 7m35s openshift-dns-operator deployment.apps/dns-operator 0/1 1 0 7m46s openshift-image-registry deployment.apps/cluster-image-registry-operator 0/1 0 0 7m5s openshift-ingress-operator deployment.apps/ingress-operator 0/1 0 0 7m5s openshift-insights deployment.apps/insights-operator 1/1 1 1 7m44s openshift-kube-apiserver-operator deployment.apps/kube-apiserver-operator 0/1 1 0 7m36s openshift-kube-controller-manager-operator deployment.apps/kube-controller-manager-operator 0/1 1 0 7m37s openshift-kube-scheduler-operator deployment.apps/openshift-kube-scheduler-operator 0/1 1 0 7m36s openshift-machine-api deployment.apps/machine-api-operator 1/1 1 1 6m46s openshift-machine-config-operator deployment.apps/machine-config-operator 1/1 1 1 7m47s openshift-marketplace deployment.apps/marketplace-operator 0/1 0 0 7m45s openshift-monitoring deployment.apps/cluster-monitoring-operator 0/1 0 0 7m39s openshift-network-operator deployment.apps/network-operator 1/1 1 1 7m48s openshift-operator-lifecycle-manager deployment.apps/catalog-operator 0/1 1 0 7m41s openshift-operator-lifecycle-manager deployment.apps/olm-operator 0/1 1 0 7m41s openshift-ovn-kubernetes deployment.apps/ovnkube-master 1/1 1 1 7m30s openshift-service-ca-operator deployment.apps/service-ca-operator 1/1 1 1 7m38s openshift-service-catalog-apiserver-operator deployment.apps/openshift-service-catalog-apiserver-operator 0/1 0 0 7m35s openshift-service-catalog-controller-manager-operator deployment.apps/openshift-service-catalog-controller-manager-operator 0/1 0 0 7m36s openshift-vertical-pod-autoscaler deployment.apps/vertical-pod-autoscaler-operator 1/1 1 1 7m42s NAMESPACE NAME DESIRED CURRENT READY AGE openshift-apiserver-operator replicaset.apps/openshift-apiserver-operator-5c64784d47 1 1 0 7m36s openshift-authentication-operator replicaset.apps/authentication-operator-6fc4b64446 1 0 0 7m8s openshift-cloud-credential-operator replicaset.apps/cloud-credential-operator-75fcf9d7f6 1 1 1 7m46s openshift-cluster-machine-approver replicaset.apps/machine-approver-5fb9454f5b 1 1 1 7m48s openshift-cluster-node-tuning-operator replicaset.apps/cluster-node-tuning-operator-6d64dbb9fc 1 0 0 7m47s openshift-cluster-storage-operator replicaset.apps/cluster-storage-operator-75d8554649 1 0 0 7m47s openshift-cluster-version replicaset.apps/cluster-version-operator-79996b9db4 1 1 1 7m29s openshift-cluster-version replicaset.apps/cluster-version-operator-7bd9f465c4 0 0 0 8m29s openshift-controller-manager-operator replicaset.apps/openshift-controller-manager-operator-76cf7864d9 1 1 0 7m35s openshift-dns-operator replicaset.apps/dns-operator-67ffdc648b 1 1 0 4m22s openshift-dns-operator replicaset.apps/dns-operator-b5b485b4d 0 0 0 7m46s openshift-image-registry replicaset.apps/cluster-image-registry-operator-784574746d 1 0 0 7m5s openshift-ingress-operator replicaset.apps/ingress-operator-7f79b89c57 1 0 0 7m5s openshift-insights replicaset.apps/insights-operator-64c548fbfb 1 1 1 7m43s openshift-insights replicaset.apps/insights-operator-7b6c67955 0 0 0 7m41s openshift-kube-apiserver-operator replicaset.apps/kube-apiserver-operator-65c654d984 1 1 0 7m36s openshift-kube-controller-manager-operator replicaset.apps/kube-controller-manager-operator-8685fddd78 1 1 0 7m36s openshift-kube-scheduler-operator replicaset.apps/openshift-kube-scheduler-operator-5994687fb 1 1 0 7m36s openshift-machine-api replicaset.apps/machine-api-operator-766c9b669c 1 1 1 6m46s openshift-machine-config-operator replicaset.apps/machine-config-operator-57b8d4db67 1 1 1 7m47s openshift-marketplace replicaset.apps/marketplace-operator-98bb96567 1 0 0 7m45s openshift-monitoring replicaset.apps/cluster-monitoring-operator-dd65594f9 1 0 0 7m39s openshift-network-operator replicaset.apps/network-operator-6f5fb67c68 1 1 1 7m48s openshift-operator-lifecycle-manager replicaset.apps/catalog-operator-6dd75c7d87 1 1 0 7m41s openshift-operator-lifecycle-manager replicaset.apps/olm-operator-854cffcff7 1 1 0 7m41s openshift-ovn-kubernetes replicaset.apps/ovnkube-master-7fb7c59fd6 1 1 1 7m29s openshift-service-ca-operator replicaset.apps/service-ca-operator-667d77b8b9 1 1 1 7m38s openshift-service-catalog-apiserver-operator replicaset.apps/openshift-service-catalog-apiserver-operator-96f649554 1 0 0 7m35s openshift-service-catalog-controller-manager-operator replicaset.apps/openshift-service-catalog-controller-manager-operator-79c97695c4 1 0 0 7m36s openshift-vertical-pod-autoscaler replicaset.apps/vertical-pod-autoscaler-operator-5fc74c8579 1 1 1 7m42s Glenns-MacBook-Pro-2:gw.lo gwest$
DEBUG OpenShift Installer unreleased-master-1508-g5e9cde9c162b313eed46ad964c9211f90e633681-dirty Glenns-MacBook-Pro-2:gw.lo gwest$ ./clusteroperatorstatus.sh apiVersion: v1 items: - apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: "2019-08-05T20:29:33Z" generation: 1 name: network resourceVersion: "2258" selfLink: /apis/config.openshift.io/v1/clusteroperators/network uid: bc437148-b7bf-11e9-b235-0050561f2525 spec: {} status: conditions: - lastTransitionTime: "2019-08-05T20:29:34Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T20:29:34Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T20:29:34Z" message: DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T20:29:34Z" message: The network is starting up reason: Startup status: "False" type: Available extension: null
openshift-install unreleased-master-1508-g5e9cde9c162b313eed46ad964c9211f90e633681-dirty built from commit 5e9cde9c162b313eed46ad964c9211f90e633681 release image registry.svc.ci.openshift.org/ocp/release@sha256:c8384a27c7c785cfeed564f9fc0373a132b25fb83a5d72d29afdf477518a42ab Glenns-MacBook-Pro-2:gw.lo gwest$ Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all NAME READY STATUS RESTARTS AGE pod/multus-267bc 1/1 Running 0 23m pod/multus-admission-controller-7tkgl 0/1 ContainerCreating 0 23m pod/multus-admission-controller-h6846 0/1 ContainerCreating 0 23m pod/multus-admission-controller-ztfdm 0/1 ContainerCreating 0 23m pod/multus-dtrzk 1/1 Running 0 23m pod/multus-f8gwk 1/1 Running 0 23m pod/multus-fqnq6 1/1 Running 0 23m pod/multus-r7tzd 1/1 Running 0 23m pod/multus-sf7tq 1/1 Running 0 23m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/multus-admission-controller ClusterIP 172.30.97.143 <none> 443/TCP 23m NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/multus 6 6 6 6 6 kubernetes.io/os=linux 23m daemonset.apps/multus-admission-controller 3 3 0 3 0 node-role.kubernetes.io/master= 23m Exploring cultus not starting: Glenns-MacBook-Pro-2:gw.lo gwest$ oc logs pod/multus-admission-controller-7tkgl Error from server (BadRequest): container "multus-admission-controller" in pod "multus-admission-controller-7tkgl" is waiting to start: ContainerCreating Glenns-MacBook-Pro-2:gw.lo gwest$ oc describe pod/multus-admission-controller-7tkgl Name: multus-admission-controller-7tkgl Namespace: openshift-multus Priority: 2000000000 PriorityClassName: system-cluster-critical Node: control-plane-2/192.168.1.203 Start Time: Mon, 05 Aug 2019 15:29:37 -0500 Labels: app=multus-admission-controller component=network controller-revision-hash=5fd5775d87 namespace=openshift-multus openshift.io/component=network pod-template-generation=1 type=infra Annotations: ovn: {"ip_address":"10.131.0.3/23", "mac_address":"ee:a5:d1:83:00:04", "gateway_ip": "10.131.0.1"} Status: Pending IP: Controlled By: DaemonSet/multus-admission-controller Containers: multus-admission-controller: Container ID: Image: registry.svc.ci.openshift.org/ocp/4.2-2019-08-05-175235@sha256:d141522c3c19f598a50765dcdbcd56f70c4c3ccae0004d5922908eebba7b6205 Image ID: Port: <none> Host Port: <none> Command: /usr/bin/webhook Args: -bind-address=0.0.0.0 -port=6443 -tls-private-key-file=/etc/webhook/tls.key -tls-cert-file=/etc/webhook/tls.crt -alsologtostderr=true State: Waiting Reason: ContainerCreating Ready: False Restart Count: 0 Requests: cpu: 10m Environment: <none> Mounts: /etc/webhook from webhook-certs (ro) /var/run/secrets/kubernetes.io/serviceaccount from multus-token-vslwd (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: webhook-certs: Type: Secret (a volume populated by a Secret) SecretName: multus-admission-controller-secret Optional: false multus-token-vslwd: Type: Secret (a volume populated by a Secret) SecretName: multus-token-vslwd Optional: false QoS Class: Burstable Node-Selectors: node-role.kubernetes.io/master= Tolerations: node-role.kubernetes.io/master:NoSchedule node.kubernetes.io/disk-pressure:NoSchedule node.kubernetes.io/memory-pressure:NoSchedule node.kubernetes.io/not-ready:NoSchedule node.kubernetes.io/not-ready:NoExecute node.kubernetes.io/pid-pressure:NoSchedule node.kubernetes.io/unreachable:NoExecute node.kubernetes.io/unschedulable:NoSchedule Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 21m default-scheduler Successfully assigned openshift-multus/multus-admission-controller-7tkgl to control-plane-2 Warning NetworkNotReady 21m (x18 over 21m) kubelet, control-plane-2 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Warning FailedMount 6m42s (x6 over 18m) kubelet, control-plane-2 Unable to mount volumes for pod "multus-admission-controller-7tkgl_openshift-multus(be71faf6-b7bf-11e9-b235-0050561f2525)": timeout expired waiting for volumes to attach or mount for pod "openshift-multus"/"multus-admission-controller-7tkgl". list of unmounted volumes=[webhook-certs]. list of unattached volumes=[webhook-certs multus-token-vslwd] Warning FailedMount 83s (x18 over 21m) kubelet, control-plane-2 MountVolume.SetUp failed for volume "webhook-certs" : secrets "multus-admission-controller-secret" not found Glenns-MacBook-Pro-2:gw.lo gwest$
Glenns-MacBook-Pro-2:gw.lo gwest$ openshift-install version openshift-install unreleased-master-1508-g5e9cde9c162b313eed46ad964c9211f90e633681-dirty built from commit 5e9cde9c162b313eed46ad964c9211f90e633681 release image registry.svc.ci.openshift.org/ocp/release@sha256:6ca9b56129b2a67686c6157693038ea9b2c227e77a9def1b962aad9e79080b76 Glenns-MacBook-Pro-2:gw.lo gwest$ Glenns-MacBook-Pro-2:gw.lo gwest$ oc project openshift-multus Now using project "openshift-multus" on server "https://api.gw.lo:6443". Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all NAME READY STATUS RESTARTS AGE pod/multus-22hfr 1/1 Running 0 5m44s pod/multus-2fjtj 1/1 Running 0 5m44s pod/multus-4pfhs 1/1 Running 0 5m44s pod/multus-6zmnt 1/1 Running 0 5m44s pod/multus-admission-controller-8kjn5 0/1 ContainerCreating 0 5m42s pod/multus-admission-controller-nnkrx 0/1 ContainerCreating 0 5m42s pod/multus-admission-controller-ztx78 0/1 ContainerCreating 0 5m42s pod/multus-g2ckc 1/1 Running 0 5m44s pod/multus-sb7rf 1/1 Running 0 5m44s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/multus-admission-controller ClusterIP 172.30.166.59 <none> 443/TCP 5m44s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/multus 6 6 6 6 6 kubernetes.io/os=linux 5m44s daemonset.apps/multus-admission-controller 3 3 0 3 0 node-role.kubernetes.io/master= 5m42s Glenns-MacBook-Pro-2:gw.lo gwest$ oc describe pod/multus-admission-controller-8kjn5 Name: multus-admission-controller-8kjn5 Namespace: openshift-multus Priority: 2000000000 PriorityClassName: system-cluster-critical Node: control-plane-0/192.168.1.201 Start Time: Mon, 05 Aug 2019 16:29:17 -0500 Labels: app=multus-admission-controller component=network controller-revision-hash=6f589d4f69 namespace=openshift-multus openshift.io/component=network pod-template-generation=1 type=infra Annotations: ovn: {"ip_address":"10.129.2.3/23", "mac_address":"0a:30:77:81:02:04", "gateway_ip": "10.129.2.1"} Status: Pending IP: Controlled By: DaemonSet/multus-admission-controller Containers: multus-admission-controller: Container ID: Image: registry.svc.ci.openshift.org/ocp/4.2-2019-08-05-184332@sha256:d141522c3c19f598a50765dcdbcd56f70c4c3ccae0004d5922908eebba7b6205 Image ID: Port: <none> Host Port: <none> Command: /usr/bin/webhook Args: -bind-address=0.0.0.0 -port=6443 -tls-private-key-file=/etc/webhook/tls.key -tls-cert-file=/etc/webhook/tls.crt -alsologtostderr=true State: Waiting Reason: ContainerCreating Ready: False Restart Count: 0 Requests: cpu: 10m Environment: <none> Mounts: /etc/webhook from webhook-certs (ro) /var/run/secrets/kubernetes.io/serviceaccount from multus-token-smw8t (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: webhook-certs: Type: Secret (a volume populated by a Secret) SecretName: multus-admission-controller-secret Optional: false multus-token-smw8t: Type: Secret (a volume populated by a Secret) SecretName: multus-token-smw8t Optional: false QoS Class: Burstable Node-Selectors: node-role.kubernetes.io/master= Tolerations: node-role.kubernetes.io/master:NoSchedule node.kubernetes.io/disk-pressure:NoSchedule node.kubernetes.io/memory-pressure:NoSchedule node.kubernetes.io/not-ready:NoSchedule node.kubernetes.io/not-ready:NoExecute node.kubernetes.io/pid-pressure:NoSchedule node.kubernetes.io/unreachable:NoExecute node.kubernetes.io/unschedulable:NoSchedule Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 5m57s default-scheduler Successfully assigned openshift-multus/multus-admission-controller-8kjn5 to control-plane-0 Warning NetworkNotReady 5m23s (x18 over 5m57s) kubelet, control-plane-0 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Warning FailedMount 5m23s (x7 over 5m56s) kubelet, control-plane-0 MountVolume.SetUp failed for volume "webhook-certs" : secrets "multus-admission-controller-secret" not found Glenns-MacBook-Pro-2:gw.lo gwest$
EBUG OpenShift Installer unreleased-master-1508-g5e9cde9c162b313eed46ad964c9211f90e633681-dirty DEBUG Built from commit 5e9cde9c162b313eed46ad964c9211f90e633681 Glenns-MacBook-Pro-2:gw.lo gwest$ oc get nodes NAME STATUS ROLES AGE VERSION control-plane-0 Ready master 5m2s v1.14.0+506b94723 control-plane-1 Ready master 5m2s v1.14.0+506b94723 control-plane-2 Ready master 5m2s v1.14.0+506b94723 worker-0 Ready worker 5m9s v1.14.0+506b94723 worker-1 Ready worker 5m2s v1.14.0+506b94723 worker-2 Ready worker 5m2s v1.14.0+506b94723 Glenns-MacBook-Pro-2:gw.lo gwest$ NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE openshift-multus daemonset.apps/multus 6 6 6 6 6 kubernetes.io/os=linux 5m20s openshift-multus daemonset.apps/multus-admission-controller 3 3 0 3 0 node-role.kubernetes.io/master= 5m18s openshift-ovn-kubernetes daemonset.apps/ovnkube-node 6 6 6 6 6 beta.kubernetes.io/os=linux 5m13s Glenns-MacBook-Pro-2:gw.lo gwest$ oc project openshift-multus Now using project "openshift-multus" on server "https://api.gw.lo:6443". Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all NAME READY STATUS RESTARTS AGE pod/multus-2mxtv 1/1 Running 0 6m2s pod/multus-54kwg 1/1 Running 0 6m2s pod/multus-72wnn 1/1 Running 0 6m2s pod/multus-admission-controller-9pqgs 0/1 ContainerCreating 0 6m pod/multus-admission-controller-c2ng4 0/1 ContainerCreating 0 6m pod/multus-admission-controller-cdqqx 0/1 ContainerCreating 0 6m pod/multus-klpkt 1/1 Running 0 6m2s pod/multus-l5g92 1/1 Running 0 6m2s pod/multus-swtlt 1/1 Running 0 6m2s Glenns-MacBook-Pro-2:gw.lo gwest$ oc describe pod/multus-admission-controller-9pqgs Name: multus-admission-controller-9pqgs Namespace: openshift-multus Priority: 2000000000 PriorityClassName: system-cluster-critical Node: control-plane-0/192.168.1.201 Start Time: Mon, 05 Aug 2019 17:17:06 -0500 Labels: app=multus-admission-controller component=network controller-revision-hash=dcc8976bb namespace=openshift-multus openshift.io/component=network pod-template-generation=1 type=infra Annotations: ovn: {"ip_address":"10.128.2.3/23", "mac_address":"6e:a6:fa:80:02:04", "gateway_ip": "10.128.2.1"} Status: Pending IP: Controlled By: DaemonSet/multus-admission-controller Containers: multus-admission-controller: Container ID: Image: registry.svc.ci.openshift.org/ocp/4.2-2019-08-05-191858@sha256:d141522c3c19f598a50765dcdbcd56f70c4c3ccae0004d5922908eebba7b6205 Image ID: Port: <none> Host Port: <none> Command: /usr/bin/webhook Args: -bind-address=0.0.0.0 -port=6443 -tls-private-key-file=/etc/webhook/tls.key -tls-cert-file=/etc/webhook/tls.crt -alsologtostderr=true State: Waiting Reason: ContainerCreating Ready: False Restart Count: 0 Requests: cpu: 10m Environment: <none> Mounts: /etc/webhook from webhook-certs (ro) /var/run/secrets/kubernetes.io/serviceaccount from multus-token-wgdkd (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: webhook-certs: Type: Secret (a volume populated by a Secret) SecretName: multus-admission-controller-secret Optional: false multus-token-wgdkd: Type: Secret (a volume populated by a Secret) SecretName: multus-token-wgdkd Optional: false QoS Class: Burstable Node-Selectors: node-role.kubernetes.io/master= Tolerations: node-role.kubernetes.io/master:NoSchedule node.kubernetes.io/disk-pressure:NoSchedule node.kubernetes.io/memory-pressure:NoSchedule node.kubernetes.io/not-ready:NoSchedule node.kubernetes.io/not-ready:NoExecute node.kubernetes.io/pid-pressure:NoSchedule node.kubernetes.io/unreachable:NoExecute node.kubernetes.io/unschedulable:NoSchedule Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 6m36s default-scheduler Successfully assigned openshift-multus/multus-admission-controller-9pqgs to control-plane-0 Warning NetworkNotReady 6m2s (x18 over 6m36s) kubelet, control-plane-0 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Warning FailedMount 6m2s (x7 over 6m35s) kubelet, control-plane-0 MountVolume.SetUp failed for volume "webhook-certs" : secrets "multus-admission-controller-secret" not found Warning FailedMount 38s (x2 over 2m52s) kubelet, control-plane-0 Unable to mount volumes for pod "multus-admission-controller-9pqgs_openshift-multus(c29b81f3-b7ce-11e9-a5d3-0050561f2525)": timeout expired waiting for volumes to attach or mount for pod "openshift-multus"/"multus-admission-controller-9pqgs". list of unmounted volumes=[webhook-certs]. list of unattached volumes=[webhook-certs multus-token-wgdkd] Glenns-MacBook-Pro-2:gw.lo gwest$ Glenns-MacBook-Pro-2:gw.lo gwest$ oc logs pod/network-operator-cc6d95d6c-gnkvq 2019/08/05 22:17:02 Go Version: go1.10.8 2019/08/05 22:17:02 Go OS/Arch: linux/amd64 2019/08/05 22:17:02 operator-sdk Version: v0.4.1 2019/08/05 22:17:02 overriding kubernetes api to https://api-int.gw.lo:6443 2019/08/05 22:17:02 Registering Components. 2019/08/05 22:17:02 Configuring Controllers 2019/08/05 22:17:02 Starting the Cmd. 2019/08/05 22:17:03 Reconciling Network.config.openshift.io cluster 2019/08/05 22:17:03 reconciling (operator.openshift.io/v1, Kind=Network) /cluster 2019/08/05 22:17:03 does not exist, creating (operator.openshift.io/v1, Kind=Network) /cluster 2019/08/05 22:17:03 Reconciling Network.operator.openshift.io cluster 2019/08/05 22:17:03 Detected uplink MTU 1500 2019/08/05 22:17:03 Starting render phase 2019/08/05 22:17:03 successfully created (operator.openshift.io/v1, Kind=Network) /cluster 2019/08/05 22:17:03 Render phase done, rendered 25 objects 2019/08/05 22:17:03 reconciling (/v1, Kind=ConfigMap) openshift-network-operator/applied-cluster 2019/08/05 22:17:03 does not exist, creating (/v1, Kind=ConfigMap) openshift-network-operator/applied-cluster 2019/08/05 22:17:03 Reconciling update for openshift-service-ca from openshift-network-operator/cluster 2019/08/05 22:17:03 ConfigMap "openshift-service-ca" not found 2019/08/05 22:17:03 successfully created (/v1, Kind=ConfigMap) openshift-network-operator/applied-cluster 2019/08/05 22:17:03 reconciling (apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition) /network-attachment-definitions.k8s.cni.cncf.io 2019/08/05 22:17:03 does not exist, creating (apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition) /network-attachment-definitions.k8s.cni.cncf.io 2019/08/05 22:17:03 Created ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:03Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:03Z" status: "True" type: Upgradeable 2019/08/05 22:17:03 successfully created (apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition) /network-attachment-definitions.k8s.cni.cncf.io 2019/08/05 22:17:03 reconciling (/v1, Kind=Namespace) /openshift-multus 2019/08/05 22:17:03 does not exist, creating (/v1, Kind=Namespace) /openshift-multus 2019/08/05 22:17:03 successfully created (/v1, Kind=Namespace) /openshift-multus 2019/08/05 22:17:03 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus 2019/08/05 22:17:03 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus 2019/08/05 22:17:03 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus 2019/08/05 22:17:03 reconciling (/v1, Kind=ServiceAccount) openshift-multus/multus 2019/08/05 22:17:03 does not exist, creating (/v1, Kind=ServiceAccount) openshift-multus/multus 2019/08/05 22:17:03 successfully created (/v1, Kind=ServiceAccount) openshift-multus/multus 2019/08/05 22:17:03 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus 2019/08/05 22:17:03 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus 2019/08/05 22:17:03 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus 2019/08/05 22:17:03 reconciling (apps/v1, Kind=DaemonSet) openshift-multus/multus 2019/08/05 22:17:04 does not exist, creating (apps/v1, Kind=DaemonSet) openshift-multus/multus 2019/08/05 22:17:04 Reconciling update for openshift-service-ca from openshift-network-operator/cluster 2019/08/05 22:17:04 ConfigMap "openshift-service-ca" not found 2019/08/05 22:17:04 successfully created (apps/v1, Kind=DaemonSet) openshift-multus/multus 2019/08/05 22:17:04 reconciling (/v1, Kind=Service) openshift-multus/multus-admission-controller 2019/08/05 22:17:04 Reconciling update to openshift-multus/multus 2019/08/05 22:17:04 Error getting DaemonSet "openshift-multus/multus-admission-controller": DaemonSet.apps "multus-admission-controller" not found 2019/08/05 22:17:04 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:04 Error getting Deployment "openshift-ovn-kubernetes/ovnkube-master": Deployment.apps "ovnkube-master" not found 2019/08/05 22:17:04 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable 2019/08/05 22:17:04 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not yet scheduled on any nodes Waiting for DaemonSet "openshift-multus/multus-admission-controller" to be created Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Waiting for Deployment "openshift-ovn-kubernetes/ovnkube-master" to be created reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:04 Reconciling update to openshift-multus/multus 2019/08/05 22:17:04 Error getting DaemonSet "openshift-multus/multus-admission-controller": DaemonSet.apps "multus-admission-controller" not found 2019/08/05 22:17:04 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:04 Error getting Deployment "openshift-ovn-kubernetes/ovnkube-master": Deployment.apps "ovnkube-master" not found 2019/08/05 22:17:04 does not exist, creating (/v1, Kind=Service) openshift-multus/multus-admission-controller 2019/08/05 22:17:04 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" update is rolling out (4 out of 6 updated) Waiting for DaemonSet "openshift-multus/multus-admission-controller" to be created Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Waiting for Deployment "openshift-ovn-kubernetes/ovnkube-master" to be created reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:04 Reconciling update to openshift-multus/multus 2019/08/05 22:17:04 Error getting DaemonSet "openshift-multus/multus-admission-controller": DaemonSet.apps "multus-admission-controller" not found 2019/08/05 22:17:04 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:04 Error getting Deployment "openshift-ovn-kubernetes/ovnkube-master": Deployment.apps "ovnkube-master" not found 2019/08/05 22:17:04 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) Waiting for DaemonSet "openshift-multus/multus-admission-controller" to be created Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Waiting for Deployment "openshift-ovn-kubernetes/ovnkube-master" to be created reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:04 successfully created (/v1, Kind=Service) openshift-multus/multus-admission-controller 2019/08/05 22:17:04 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus-admission-controller-webhook 2019/08/05 22:17:05 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus-admission-controller-webhook 2019/08/05 22:17:05 Reconciling update for openshift-service-ca from openshift-network-operator/cluster 2019/08/05 22:17:05 ConfigMap "openshift-service-ca" not found 2019/08/05 22:17:06 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus-admission-controller-webhook 2019/08/05 22:17:06 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus-admission-controller-webhook 2019/08/05 22:17:06 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus-admission-controller-webhook 2019/08/05 22:17:06 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus-admission-controller-webhook 2019/08/05 22:17:06 reconciling (admissionregistration.k8s.io/v1beta1, Kind=ValidatingWebhookConfiguration) /multus.openshift.io 2019/08/05 22:17:06 does not exist, creating (admissionregistration.k8s.io/v1beta1, Kind=ValidatingWebhookConfiguration) /multus.openshift.io 2019/08/05 22:17:06 successfully created (admissionregistration.k8s.io/v1beta1, Kind=ValidatingWebhookConfiguration) /multus.openshift.io 2019/08/05 22:17:06 reconciling (/v1, Kind=ConfigMap) openshift-network-operator/openshift-service-ca 2019/08/05 22:17:06 does not exist, creating (/v1, Kind=ConfigMap) openshift-network-operator/openshift-service-ca 2019/08/05 22:17:06 Reconciling update for openshift-service-ca from openshift-network-operator/cluster 2019/08/05 22:17:06 ConfigMap "openshift-service-ca" not found 2019/08/05 22:17:06 successfully created (/v1, Kind=ConfigMap) openshift-network-operator/openshift-service-ca 2019/08/05 22:17:06 reconciling (apps/v1, Kind=DaemonSet) openshift-multus/multus-admission-controller 2019/08/05 22:17:06 does not exist, creating (apps/v1, Kind=DaemonSet) openshift-multus/multus-admission-controller 2019/08/05 22:17:06 Reconciling update to openshift-multus/multus-admission-controller 2019/08/05 22:17:06 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:06 Error getting Deployment "openshift-ovn-kubernetes/ovnkube-master": Deployment.apps "ovnkube-master" not found 2019/08/05 22:17:06 successfully created (apps/v1, Kind=DaemonSet) openshift-multus/multus-admission-controller 2019/08/05 22:17:06 reconciling (/v1, Kind=Namespace) /openshift-ovn-kubernetes 2019/08/05 22:17:06 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not yet scheduled on any nodes Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Waiting for Deployment "openshift-ovn-kubernetes/ovnkube-master" to be created reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:06 Reconciling update to openshift-multus/multus-admission-controller 2019/08/05 22:17:06 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:06 Error getting Deployment "openshift-ovn-kubernetes/ovnkube-master": Deployment.apps "ovnkube-master" not found 2019/08/05 22:17:07 does not exist, creating (/v1, Kind=Namespace) /openshift-ovn-kubernetes 2019/08/05 22:17:07 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" update is rolling out (1 out of 3 updated) Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Waiting for Deployment "openshift-ovn-kubernetes/ovnkube-master" to be created reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:07 Reconciling update to openshift-multus/multus-admission-controller 2019/08/05 22:17:07 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:07 Error getting Deployment "openshift-ovn-kubernetes/ovnkube-master": Deployment.apps "ovnkube-master" not found 2019/08/05 22:17:07 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Waiting for Deployment "openshift-ovn-kubernetes/ovnkube-master" to be created reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:07 Reconciling update for openshift-service-ca from openshift-network-operator/cluster 2019/08/05 22:17:07 successfully created (/v1, Kind=Namespace) /openshift-ovn-kubernetes 2019/08/05 22:17:07 reconciling (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-node 2019/08/05 22:17:07 does not exist, creating (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-node 2019/08/05 22:17:07 successfully created (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-node 2019/08/05 22:17:07 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-node 2019/08/05 22:17:07 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-node 2019/08/05 22:17:07 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-node 2019/08/05 22:17:07 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-node 2019/08/05 22:17:08 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-node 2019/08/05 22:17:08 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-node 2019/08/05 22:17:08 reconciling (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-controller 2019/08/05 22:17:08 does not exist, creating (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-controller 2019/08/05 22:17:08 successfully created (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-controller 2019/08/05 22:17:08 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-controller 2019/08/05 22:17:09 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-controller 2019/08/05 22:17:09 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-controller 2019/08/05 22:17:09 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-controller 2019/08/05 22:17:09 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-controller 2019/08/05 22:17:09 successfully created (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-controller 2019/08/05 22:17:09 reconciling (rbac.authorization.k8s.io/v1, Kind=Role) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:17:09 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=Role) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:17:09 successfully created (rbac.authorization.k8s.io/v1, Kind=Role) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:17:09 reconciling (rbac.authorization.k8s.io/v1, Kind=RoleBinding) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:17:10 does not exist, creating (rbac.authorization.k8s.io/v1, Kind=RoleBinding) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:17:10 successfully created (rbac.authorization.k8s.io/v1, Kind=RoleBinding) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:17:10 reconciling (/v1, Kind=ConfigMap) openshift-ovn-kubernetes/ovn-config 2019/08/05 22:17:10 does not exist, creating (/v1, Kind=ConfigMap) openshift-ovn-kubernetes/ovn-config 2019/08/05 22:17:10 successfully created (/v1, Kind=ConfigMap) openshift-ovn-kubernetes/ovn-config 2019/08/05 22:17:10 reconciling (/v1, Kind=Service) openshift-ovn-kubernetes/ovnkube-db 2019/08/05 22:17:10 does not exist, creating (/v1, Kind=Service) openshift-ovn-kubernetes/ovnkube-db 2019/08/05 22:17:11 successfully created (/v1, Kind=Service) openshift-ovn-kubernetes/ovnkube-db 2019/08/05 22:17:11 reconciling (apps/v1, Kind=Deployment) openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:11 does not exist, creating (apps/v1, Kind=Deployment) openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:11 Reconciling update to openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:11 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:11 successfully created (apps/v1, Kind=Deployment) openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:11 reconciling (apps/v1, Kind=DaemonSet) openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:17:11 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) Waiting for DaemonSet "openshift-ovn-kubernetes/ovnkube-node" to be created Deployment "openshift-ovn-kubernetes/ovnkube-master" is not yet scheduled on any nodes reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:11 does not exist, creating (apps/v1, Kind=DaemonSet) openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:17:11 Reconciling update to openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:11 Error getting DaemonSet "openshift-ovn-kubernetes/ovnkube-node": DaemonSet.apps "ovnkube-node" not found 2019/08/05 22:17:12 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:17:12 successfully created (apps/v1, Kind=DaemonSet) openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:17:12 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not yet scheduled on any nodes Deployment "openshift-ovn-kubernetes/ovnkube-master" is not yet scheduled on any nodes reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:12 Reconciling update to openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:12 Failed to update ClusterOperator "network": Operation cannot be fulfilled on clusteroperators.config.openshift.io "network": the object has been modified; please apply your changes to the latest version and try again 2019/08/05 22:17:12 reconciling (config.openshift.io/v1, Kind=Network) /cluster 2019/08/05 22:17:13 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not yet scheduled on any nodes Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:13 Reconciling Network.config.openshift.io cluster 2019/08/05 22:17:14 update was successful 2019/08/05 22:17:14 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:17:16 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" update is rolling out (3 out of 6 updated) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:16 Reconciling update to openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:17:17 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 6 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:17 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:17:29 Reconciling update to openshift-multus/multus 2019/08/05 22:17:29 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 5 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:30 Reconciling update to openshift-multus/multus 2019/08/05 22:17:30 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 4 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:32 Reconciling update to openshift-multus/multus 2019/08/05 22:17:32 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 3 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:17:57 Reconciling update to openshift-multus/multus 2019/08/05 22:17:57 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 2 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:02 Reconciling update to openshift-multus/multus 2019/08/05 22:18:02 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus" is not available (awaiting 1 nodes) DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:04 Reconciling update to openshift-multus/multus 2019/08/05 22:18:05 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) Deployment "openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:26 Reconciling update to openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:18:26 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 6 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:46 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:18:46 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 5 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:46 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:18:47 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 4 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:47 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:18:47 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 3 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:48 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:18:48 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 2 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:50 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:18:50 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: |- DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) DaemonSet "openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 1 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:18:51 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:18:51 Updated ClusterOperator with conditions: - lastTransitionTime: "2019-08-05T22:17:04Z" status: "False" type: Degraded - lastTransitionTime: "2019-08-05T22:17:04Z" status: "True" type: Upgradeable - lastTransitionTime: "2019-08-05T22:17:04Z" message: DaemonSet "openshift-multus/multus-admission-controller" is not available (awaiting 3 nodes) reason: Deploying status: "True" type: Progressing - lastTransitionTime: "2019-08-05T22:17:04Z" message: The network is starting up reason: Startup status: "False" type: Available 2019/08/05 22:22:04 Reconciling update to openshift-multus/multus 2019/08/05 22:22:06 Reconciling update to openshift-multus/multus-admission-controller 2019/08/05 22:22:11 Reconciling update to openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:22:12 Reconciling update to openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:22:14 Reconciling Network.operator.openshift.io cluster 2019/08/05 22:22:14 Starting render phase 2019/08/05 22:22:14 Render phase done, rendered 25 objects 2019/08/05 22:22:14 reconciling (/v1, Kind=ConfigMap) openshift-network-operator/applied-cluster 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition) /network-attachment-definitions.k8s.cni.cncf.io 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (/v1, Kind=Namespace) /openshift-multus 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (/v1, Kind=ServiceAccount) openshift-multus/multus 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (apps/v1, Kind=DaemonSet) openshift-multus/multus 2019/08/05 22:22:14 update was successful 2019/08/05 22:22:14 reconciling (/v1, Kind=Service) openshift-multus/multus-admission-controller 2019/08/05 22:22:15 update was successful 2019/08/05 22:22:15 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /multus-admission-controller-webhook 2019/08/05 22:22:15 update was successful 2019/08/05 22:22:15 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /multus-admission-controller-webhook 2019/08/05 22:22:16 update was successful 2019/08/05 22:22:16 reconciling (admissionregistration.k8s.io/v1beta1, Kind=ValidatingWebhookConfiguration) /multus.openshift.io 2019/08/05 22:22:16 update was successful 2019/08/05 22:22:16 reconciling (/v1, Kind=ConfigMap) openshift-network-operator/openshift-service-ca 2019/08/05 22:22:16 reconciling (apps/v1, Kind=DaemonSet) openshift-multus/multus-admission-controller 2019/08/05 22:22:17 update was successful 2019/08/05 22:22:17 reconciling (/v1, Kind=Namespace) /openshift-ovn-kubernetes 2019/08/05 22:22:17 update was successful 2019/08/05 22:22:17 reconciling (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-node 2019/08/05 22:22:17 update was successful 2019/08/05 22:22:17 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-node 2019/08/05 22:22:18 update was successful 2019/08/05 22:22:18 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-node 2019/08/05 22:22:18 update was successful 2019/08/05 22:22:18 reconciling (/v1, Kind=ServiceAccount) openshift-ovn-kubernetes/ovn-kubernetes-controller 2019/08/05 22:22:19 update was successful 2019/08/05 22:22:19 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRole) /openshift-ovn-kubernetes-controller 2019/08/05 22:22:19 update was successful 2019/08/05 22:22:19 reconciling (rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding) /openshift-ovn-kubernetes-controller 2019/08/05 22:22:19 update was successful 2019/08/05 22:22:19 reconciling (rbac.authorization.k8s.io/v1, Kind=Role) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:22:20 update was successful 2019/08/05 22:22:20 reconciling (rbac.authorization.k8s.io/v1, Kind=RoleBinding) openshift-ovn-kubernetes/openshift-ovn-kubernetes-sbdb 2019/08/05 22:22:20 update was successful 2019/08/05 22:22:20 reconciling (/v1, Kind=ConfigMap) openshift-ovn-kubernetes/ovn-config 2019/08/05 22:22:21 update was successful 2019/08/05 22:22:21 reconciling (/v1, Kind=Service) openshift-ovn-kubernetes/ovnkube-db 2019/08/05 22:22:21 update was successful 2019/08/05 22:22:21 reconciling (apps/v1, Kind=Deployment) openshift-ovn-kubernetes/ovnkube-master 2019/08/05 22:22:21 update was successful 2019/08/05 22:22:21 reconciling (apps/v1, Kind=DaemonSet) openshift-ovn-kubernetes/ovnkube-node 2019/08/05 22:22:22 update was successful Glenns-MacBook-Pro-2:gw.lo gwest$ Glenns-MacBook-Pro-2:gw.lo gwest$ oc project openshift-multus Now using project "openshift-multus" on server "https://api.gw.lo:6443". Glenns-MacBook-Pro-2:gw.lo gwest$ oc get all NAME READY STATUS RESTARTS AGE pod/multus-2mxtv 1/1 Running 0 9m55s pod/multus-54kwg 1/1 Running 0 9m55s pod/multus-72wnn 1/1 Running 0 9m55s pod/multus-admission-controller-9pqgs 0/1 ContainerCreating 0 9m53s pod/multus-admission-controller-c2ng4 0/1 ContainerCreating 0 9m53s pod/multus-admission-controller-cdqqx 0/1 ContainerCreating 0 9m53s pod/multus-klpkt 1/1 Running 0 9m55s pod/multus-l5g92 1/1 Running 0 9m55s pod/multus-swtlt 1/1 Running 0 9m55s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/multus-admission-controller ClusterIP 172.30.232.174 <none> 443/TCP 9m55s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/multus 6 6 6 6 6 kubernetes.io/os=linux 9m55s daemonset.apps/multus-admission-controller 3 3 0 3 0 node-role.kubernetes.io/master= 9m53s Glenns-MacBook-Pro-2:gw.lo gwest$ oc logs pod/multus-2mxtv 2019-08-05T22:17:56+0000 Generating Multus configuration file ... 2019-08-05T22:17:56+0000 Attemping to find master plugin configuration, attempt 0 2019-08-05T22:18:01+0000 Attemping to find master plugin configuration, attempt 5 2019-08-05T22:18:06+0000 Attemping to find master plugin configuration, attempt 10 2019-08-05T22:18:11+0000 Attemping to find master plugin configuration, attempt 15 2019-08-05T22:18:17+0000 Attemping to find master plugin configuration, attempt 20 2019-08-05T22:18:22+0000 Attemping to find master plugin configuration, attempt 25 2019-08-05T22:18:27+0000 Attemping to find master plugin configuration, attempt 30 2019-08-05T22:18:32+0000 Attemping to find master plugin configuration, attempt 35 2019-08-05T22:18:37+0000 Attemping to find master plugin configuration, attempt 40 2019-08-05T22:18:42+0000 Attemping to find master plugin configuration, attempt 45 2019-08-05T22:18:44+0000 Config file created @ /host/etc/cni/net.d/00-multus.conf { "name": "multus-cni-network", "type": "multus", "namespaceIsolation": true, "logLevel": "verbose", "kubeconfig": "/etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig", "delegates": [ {"cniVersion":"0.3.1","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay","ipam":{},"dns":{}} ] } 2019-08-05T22:18:44+0000 Entering sleep... (success) Glenns-MacBook-Pro-2:gw.lo gwest$
hi, we have another similar bug https://bugzilla.redhat.com/show_bug.cgi?id=1729091 is tracing this issue can we mark this bug as 'duplicate'.
Least from looking at the logs, its a different issue. I did look at the 1729091, and didn't see the same "markers" in this.
After reviewing it another time, I don't mind trying a nightly that passes the dns bug.. So far I've tried every ci and nightly, and none have worked. Last working ovn based image was on 7-31. As I would need to reopen, if its not duplicate, prefer not to mark it as duplicate. We have several devs waiting on this to be "working" again.
the older bug ended up tracking this as well, since they can't verify that the dns bug was fixed until this bug is fixed too... *** This bug has been marked as a duplicate of bug 1729091 ***