+++ This bug was initially created as a clone of Bug #1779654 +++ Description of problem: On some HW, the installation fails due to the startup order of operators that caused by a delay in OSP components. Debug info below Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. Deploy OSP13 2. Install OCP 4.2 with Kuryr 3. Actual results: installation failed Expected results: Installation passed Additional info: (undercloud) [stack@undercloud-0 ~]$ oc get nodes NAME STATUS ROLES AGE VERSION ostest-wg9bj-master-0 Ready master 12h v1.14.6+31a56cf75 ostest-wg9bj-master-1 Ready master 12h v1.14.6+31a56cf75 ostest-wg9bj-master-2 Ready master 12h v1.14.6+31a56cf75 ostest-wg9bj-worker-cmnlk Ready worker 11h v1.14.6+31a56cf75 ostest-wg9bj-worker-r4sj5 Ready worker 11h v1.14.6+31a56cf75 ostest-wg9bj-worker-wdmxw Ready worker 11h v1.14.6+31a56cf75 (undercloud) [stack@undercloud-0 ~]$ oc get co NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE cloud-credential 4.2.0-0.nightly-2019-11-26-164052 True False False 11h dns 4.2.0-0.nightly-2019-11-26-164052 True False False 11h insights 4.2.0-0.nightly-2019-11-26-164052 True False True 11h kube-apiserver 4.2.0-0.nightly-2019-11-26-164052 True False False 11h kube-controller-manager 4.2.0-0.nightly-2019-11-26-164052 True False False 11h kube-scheduler 4.2.0-0.nightly-2019-11-26-164052 True False False 11h machine-api 4.2.0-0.nightly-2019-11-26-164052 True False False 11h machine-config 4.2.0-0.nightly-2019-11-26-164052 True False False 11h network 4.2.0-0.nightly-2019-11-26-164052 True False False 11h openshift-controller-manager 4.2.0-0.nightly-2019-11-26-164052 True False False 11h operator-lifecycle-manager 4.2.0-0.nightly-2019-11-26-164052 True False False 11h operator-lifecycle-manager-catalog 4.2.0-0.nightly-2019-11-26-164052 True False False 11h operator-lifecycle-manager-packageserver 4.2.0-0.nightly-2019-11-26-164052 True False False 11h service-ca 4.2.0-0.nightly-2019-11-26-164052 True False False 11h (undercloud) [stack@undercloud-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version False True 12h Unable to apply 4.2.0-0.nightly-2019-11-26-164052: an unknown error has occurred (undercloud) [stack@undercloud-0 ~]$ openstack server list +--------------------------------------+--------------+--------+------------------------+----------------+------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+--------------+--------+------------------------+----------------+------------+ | 981b5fdd-54f8-42ce-83a4-439dcbb062aa | controller-1 | ACTIVE | ctlplane=192.168.24.20 | overcloud-full | controller | | 3e092d44-4de0-49bf-b348-5f0cb5642378 | controller-2 | ACTIVE | ctlplane=192.168.24.17 | overcloud-full | controller | | 694d4d76-45e1-4a4c-a02e-acd69d770b1d | compute-0 | ACTIVE | ctlplane=192.168.24.15 | overcloud-full | compute | | 6e62763e-1fbf-4254-9961-4416bec7d04a | controller-0 | ACTIVE | ctlplane=192.168.24.12 | overcloud-full | controller | +--------------------------------------+--------------+--------+------------------------+----------------+------------+ (undercloud) [stack@undercloud-0 ~]$ . shiftstackrc (shiftstack) [stack@undercloud-0 ~]$ openstack server list +--------------------------------------+---------------------------+--------+------------------------------------+-------+-----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+---------------------------+--------+------------------------------------+-------+-----------+ | c343db2e-5941-4e0e-9b0a-c510031d7dce | ostest-wg9bj-worker-r4sj5 | ACTIVE | ostest-wg9bj-openshift=10.196.0.36 | rhcos | m4.xlarge | | ae93ffae-db0c-4282-9a76-e5e1dca97e86 | ostest-wg9bj-worker-wdmxw | ACTIVE | ostest-wg9bj-openshift=10.196.0.38 | rhcos | m4.xlarge | | af9784e1-e6f5-4f44-8058-b83df6c54d40 | ostest-wg9bj-worker-cmnlk | ACTIVE | ostest-wg9bj-openshift=10.196.0.34 | rhcos | m4.xlarge | | f2a8fc24-78cc-44bf-95f4-a94775d15c63 | ostest-wg9bj-master-0 | ACTIVE | ostest-wg9bj-openshift=10.196.0.13 | rhcos | m4.xlarge | | 719a23bc-85f4-42c1-aea2-a9afc62d95a4 | ostest-wg9bj-master-2 | ACTIVE | ostest-wg9bj-openshift=10.196.0.29 | rhcos | m4.xlarge | | dd693a11-4fee-4149-a3c2-4c8717085f2d | ostest-wg9bj-master-1 | ACTIVE | ostest-wg9bj-openshift=10.196.0.15 | rhcos | m4.xlarge | +--------------------------------------+---------------------------+--------+------------------------------------+-------+-----------+ (shiftstack) [stack@undercloud-0 ~]$ oc describe pod openshift-apiserver-operator-7648756bf6-snj67 -n openshift-apiserver-operator Name: openshift-apiserver-operator-7648756bf6-snj67 Namespace: openshift-apiserver-operator Priority: 2000000000 PriorityClassName: system-cluster-critical Node: ostest-wg9bj-master-1/10.196.0.15 Start Time: Tue, 26 Nov 2019 17:20:01 -0500 Labels: app=openshift-apiserver-operator pod-template-hash=7648756bf6 Annotations: k8s.v1.cni.cncf.io/networks-status: [{ "name": "kuryr", "interface": "eth0", "ips": [ "10.128.40.9" ], "mac": "fa:16:3e:b9:fd:e6", "default": true, "dns": {} }] openstack.org/kuryr-pod-label: {"app": "openshift-apiserver-operator", "pod-template-hash": "7648756bf6"} openstack.org/kuryr-vif: {"versioned_object.changes": ["default_vif"], "versioned_object.data": {"additional_vifs": {}, "default_vif": {"versioned_object.changes":... Status: Running IP: 10.128.40.9 Controlled By: ReplicaSet/openshift-apiserver-operator-7648756bf6 Containers: openshift-apiserver-operator: Container ID: cri-o://84e9889eda25003036998b2c0bbd5d2d06ca74f5f62a690a40b4601887f10f03 Image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f04c416a210b19772d20b73d24cbe2da79f565f688eaef33029dbc2328fa4be7 Image ID: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f04c416a210b19772d20b73d24cbe2da79f565f688eaef33029dbc2328fa4be7 Port: 8443/TCP Host Port: 0/TCP Command: cluster-openshift-apiserver-operator operator Args: --config=/var/run/configmaps/config/config.yaml State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Message: I1127 10:19:30.592561 1 cmd.go:160] Using service-serving-cert provided certificates I1127 10:19:30.593211 1 observer_polling.go:106] Starting file observer W1127 10:19:33.664944 1 builder.go:181] unable to get owner reference (falling back to namespace): Get https://172.30.0.1:443/api/v1/namespaces/openshift-apiserver-operator/pods: dial tcp 172.30.0.1:443: connect: no route to host F1127 10:20:04.555120 1 cmd.go:92] Get https://172.30.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication: dial tcp 172.30.0.1:443: connect: no route to host Exit Code: 255 Started: Wed, 27 Nov 2019 05:19:30 -0500 Finished: Wed, 27 Nov 2019 05:20:04 -0500 Ready: False Restart Count: 130 Requests: cpu: 10m memory: 50Mi Environment: IMAGE: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:699ef43532027b2429eeb48b809e27c2c1b0e750b2e5fa0d84159540c141c7ca OPERATOR_IMAGE: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f04c416a210b19772d20b73d24cbe2da79f565f688eaef33029dbc2328fa4be7 OPERATOR_IMAGE_VERSION: 4.2.0-0.nightly-2019-11-26-164052 OPERAND_IMAGE_VERSION: 4.2.0-0.nightly-2019-11-26-164052 Mounts: /var/run/configmaps/config from config (rw) /var/run/secrets/kubernetes.io/serviceaccount from openshift-apiserver-operator-token-scnqv (ro) /var/run/secrets/serving-cert from serving-cert (rw) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: serving-cert: Type: Secret (a volume populated by a Secret) SecretName: openshift-apiserver-operator-serving-cert Optional: true config: Type: ConfigMap (a volume populated by a ConfigMap) Name: openshift-apiserver-operator-config Optional: false openshift-apiserver-operator-token-scnqv: Type: Secret (a volume populated by a Secret) SecretName: openshift-apiserver-operator-token-scnqv Optional: false QoS Class: Burstable Node-Selectors: node-role.kubernetes.io/master= Tolerations: node-role.kubernetes.io/master:NoSchedule node.kubernetes.io/memory-pressure:NoSchedule node.kubernetes.io/not-ready:NoExecute for 120s node.kubernetes.io/unreachable:NoExecute for 120s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Pulled 24m (x126 over 11h) kubelet, ostest-wg9bj-master-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f04c416a210b19772d20b73d24cbe2da79f565f688eaef33029dbc2328fa4be7" already present on machine Warning BackOff 4m19s (x2946 over 11h) kubelet, ostest-wg9bj-master-1 Back-off restarting failed container (shiftstack) [stack@undercloud-0 ~]$ #oc describe pod openshift-apiserver-operator-7648756bf6-snj67 -n openshift-apiserver-operator (shiftstack) [stack@undercloud-0 ~]$ oc logs openshift-apiserver-operator-7648756bf6-snj67 -n openshift-apiserver-operator I1127 10:19:30.592561 1 cmd.go:160] Using service-serving-cert provided certificates I1127 10:19:30.593211 1 observer_polling.go:106] Starting file observer W1127 10:19:33.664944 1 builder.go:181] unable to get owner reference (falling back to namespace): Get https://172.30.0.1:443/api/v1/namespaces/openshift-apiserver-operator/pods: dial tcp 172.30.0.1:443: connect: no route to host F1127 10:20:04.555120 1 cmd.go:92] Get https://172.30.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication: dial tcp 172.30.0.1:443: connect: no route to host (shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer list +--------------------------------------+-----------------------------------------------------------------------+----------------------------------+----------------+---------------------+----------+ | id | name | project_id | vip_address | provisioning_status | provider | +--------------------------------------+-----------------------------------------------------------------------+----------------------------------+----------------+---------------------+----------+ | 9f6f2c5f-4fdd-4986-8d39-1d59768206ef | ostest-wg9bj-kuryr-api-loadbalancer | aa0378b6eb544d59a84c78a790f0a768 | 172.30.0.1 | ACTIVE | octavia | | 07489b31-730e-4f2d-b7e8-83c8b6888ad2 | openshift-etcd/etcd | aa0378b6eb544d59a84c78a790f0a768 | 172.30.20.213 | ACTIVE | octavia | | 28c0891a-addb-4d34-9221-83d2cd36b985 | openshift-cluster-version/cluster-version-operator | aa0378b6eb544d59a84c78a790f0a768 | 172.30.118.219 | ACTIVE | octavia | | 9ce335fb-2565-447a-bdbb-a7be2d3f6769 | openshift-kube-scheduler-operator/metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.130.117 | ACTIVE | octavia | | 6ef04bb2-4378-43aa-b793-1f3dc2a177a6 | openshift-kube-controller-manager-operator/metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.62.162 | ACTIVE | octavia | | 1b1a1ffe-7048-457c-8e92-c8b53be2a30d | openshift-kuryr/kuryr-dns-admission-controller | aa0378b6eb544d59a84c78a790f0a768 | 172.30.62.33 | ACTIVE | octavia | | 314956c5-e8f7-4505-8118-eb9c664a45d0 | openshift-machine-api/machine-api-operator | aa0378b6eb544d59a84c78a790f0a768 | 172.30.82.129 | ACTIVE | octavia | | f35fa438-d40d-4454-ba00-046e74ad4b72 | openshift-apiserver-operator/metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.54.49 | ACTIVE | octavia | | dfc5427c-89ca-4af1-b8c6-43d9ea6b8869 | openshift-cloud-credential-operator/controller-manager-service | aa0378b6eb544d59a84c78a790f0a768 | 172.30.121.201 | ACTIVE | octavia | | 46c4535d-ef50-473f-add5-1d7613639185 | openshift-kube-apiserver-operator/metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.211.35 | ACTIVE | octavia | | 854da7fc-d095-459e-a1b8-3e7de3a36bd0 | openshift-controller-manager-operator/metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.56.92 | ACTIVE | octavia | | dbbae311-14c3-4298-a228-4df5e500275e | openshift-dns/dns-default | aa0378b6eb544d59a84c78a790f0a768 | 172.30.0.10 | ACTIVE | octavia | | 74bd7e77-97ff-4e06-89c7-f6ed68e1a988 | openshift-multus/multus-admission-controller | aa0378b6eb544d59a84c78a790f0a768 | 172.30.82.85 | ACTIVE | octavia | | 307b0eb5-ee04-467a-a548-fde874cd95a6 | openshift-kube-apiserver/apiserver | aa0378b6eb544d59a84c78a790f0a768 | 172.30.194.43 | ACTIVE | octavia | | 280204d9-5cfb-4acf-8983-1737dae5d773 | openshift-operator-lifecycle-manager/olm-operator-metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.248.45 | ACTIVE | octavia | | 9ad9135f-0663-44fb-a984-640c0e8f639e | openshift-kube-scheduler/scheduler | aa0378b6eb544d59a84c78a790f0a768 | 172.30.226.138 | ACTIVE | octavia | | 646a2ead-525f-48b2-9983-9c306b0745b1 | openshift-operator-lifecycle-manager/catalog-operator-metrics | aa0378b6eb544d59a84c78a790f0a768 | 172.30.17.72 | ACTIVE | octavia | | 1bcbd619-251f-4043-8ea2-c8dd23760dae | openshift-controller-manager/controller-manager | aa0378b6eb544d59a84c78a790f0a768 | 172.30.62.44 | ACTIVE | octavia | | e273fc88-8e1e-432c-9072-4acea769549c | openshift-kube-controller-manager/kube-controller-manager | aa0378b6eb544d59a84c78a790f0a768 | 172.30.32.62 | ACTIVE | octavia | | d035e23a-40fe-462c-af3a-98b553454e0f | openshift-operator-lifecycle-manager/v1-packages-operators-coreos-com | aa0378b6eb544d59a84c78a790f0a768 | 172.30.16.122 | ACTIVE | octavia | +--------------------------------------+-----------------------------------------------------------------------+----------------------------------+----------------+---------------------+----------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer amphora list +--------------------------------------+--------------------------------------+-----------+------------+---------------+----------------+ | id | loadbalancer_id | status | role | lb_network_ip | ha_ip | +--------------------------------------+--------------------------------------+-----------+------------+---------------+----------------+ | 09453a6a-b75e-4100-a909-b0a5bf4ac6f2 | 9f6f2c5f-4fdd-4986-8d39-1d59768206ef | ALLOCATED | STANDALONE | 172.24.0.27 | 172.30.0.1 | | 158a48ab-867b-458f-94e3-6aba65609996 | d035e23a-40fe-462c-af3a-98b553454e0f | ALLOCATED | STANDALONE | 172.24.0.30 | 172.30.16.122 | | 1cbfdbf2-0004-4bcc-a72a-dcde7bc61b26 | f35fa438-d40d-4454-ba00-046e74ad4b72 | ALLOCATED | STANDALONE | 172.24.0.14 | 172.30.54.49 | | 25e45a15-fe95-4987-a390-3181d7a008ad | 9ce335fb-2565-447a-bdbb-a7be2d3f6769 | ALLOCATED | STANDALONE | 172.24.0.6 | 172.30.130.117 | | 326c93b5-dec0-4821-bbb1-0385a81a7f07 | 6ef04bb2-4378-43aa-b793-1f3dc2a177a6 | ALLOCATED | STANDALONE | 172.24.0.17 | 172.30.62.162 | | 3a9fac66-8751-4219-9133-dd86e02ae34f | dfc5427c-89ca-4af1-b8c6-43d9ea6b8869 | ALLOCATED | STANDALONE | 172.24.0.33 | 172.30.121.201 | | 50635b26-d9b3-4377-b384-d170e11b92be | e273fc88-8e1e-432c-9072-4acea769549c | ALLOCATED | STANDALONE | 172.24.0.21 | 172.30.32.62 | | 5c2f3b69-f614-4ee0-ba00-d394488bea4b | 9ad9135f-0663-44fb-a984-640c0e8f639e | ALLOCATED | STANDALONE | 172.24.0.20 | 172.30.226.138 | | 5ff184ec-5164-4d81-9080-436c955f1797 | 314956c5-e8f7-4505-8118-eb9c664a45d0 | ALLOCATED | STANDALONE | 172.24.0.9 | 172.30.82.129 | | 61e0968a-d4ea-4bd2-bdc8-b19c99f0f844 | 74bd7e77-97ff-4e06-89c7-f6ed68e1a988 | ALLOCATED | STANDALONE | 172.24.0.19 | 172.30.82.85 | | 7264d58e-c2a1-4efe-b8c2-b6145bdea831 | dbbae311-14c3-4298-a228-4df5e500275e | ALLOCATED | STANDALONE | 172.24.0.16 | 172.30.0.10 | | 7b11dc69-71a3-43c7-b680-66150134ce65 | 07489b31-730e-4f2d-b7e8-83c8b6888ad2 | ALLOCATED | STANDALONE | 172.24.0.11 | 172.30.20.213 | | aa3c623d-be46-40fa-9731-c528ad1a744e | 307b0eb5-ee04-467a-a548-fde874cd95a6 | ALLOCATED | STANDALONE | 172.24.0.22 | 172.30.194.43 | | c3bc608d-a70a-4b14-940c-4c2ca167b93f | 646a2ead-525f-48b2-9983-9c306b0745b1 | ALLOCATED | STANDALONE | 172.24.0.12 | 172.30.17.72 | | d1ea9e02-2914-4af4-9e64-f97624d30113 | 46c4535d-ef50-473f-add5-1d7613639185 | ALLOCATED | STANDALONE | 172.24.0.38 | 172.30.211.35 | | d2d2727d-e16d-4a28-b85a-3d1406008fbc | 1b1a1ffe-7048-457c-8e92-c8b53be2a30d | ALLOCATED | STANDALONE | 172.24.0.13 | 172.30.62.33 | | f485fdbd-b0a0-4f64-b578-a1e2b4b18c33 | 28c0891a-addb-4d34-9221-83d2cd36b985 | ALLOCATED | STANDALONE | 172.24.0.15 | 172.30.118.219 | | f52cf9f9-02ee-4a8c-9451-4c4e24f7d629 | 1bcbd619-251f-4043-8ea2-c8dd23760dae | ALLOCATED | STANDALONE | 172.24.0.42 | 172.30.62.44 | | f90b5b9f-7dce-4a9d-859c-d414e0499088 | 854da7fc-d095-459e-a1b8-3e7de3a36bd0 | ALLOCATED | STANDALONE | 172.24.0.24 | 172.30.56.92 | | fa6ba4d4-0df5-4934-a10c-175e75b0743c | 280204d9-5cfb-4acf-8983-1737dae5d773 | ALLOCATED | STANDALONE | 172.24.0.10 | 172.30.248.45 | +--------------------------------------+--------------------------------------+-----------+------------+---------------+----------------+ (overcloud) [stack@undercloud-0 ~]$ oc get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE openshift-apiserver-operator openshift-apiserver-operator-7648756bf6-snj67 1/1 Running 131 12h openshift-cloud-credential-operator cloud-credential-operator-cd56bbfd6-kgp9b 1/1 Running 1 12h openshift-cluster-machine-approver machine-approver-f99ddd99f-bst6c 1/1 Running 0 12h openshift-cluster-version cluster-version-operator-865b5b8d46-gf2j4 1/1 Running 1 12h openshift-controller-manager-operator openshift-controller-manager-operator-58f688fbd9-kvxg2 1/1 Running 2 12h openshift-controller-manager controller-manager-fhzxf 1/1 Running 35 12h openshift-controller-manager controller-manager-r7xtx 1/1 Running 40 12h openshift-controller-manager controller-manager-wrf2b 1/1 Running 49 12h openshift-dns-operator dns-operator-d864b4d65-5g68h 1/1 Running 0 12h openshift-dns dns-default-j7n9t 2/2 Running 0 11h openshift-dns dns-default-krv6q 2/2 Running 0 12h openshift-dns dns-default-m6k72 2/2 Running 0 11h openshift-dns dns-default-sgxs7 2/2 Running 0 12h openshift-dns dns-default-vl9t7 2/2 Running 0 12h openshift-dns dns-default-w65sk 2/2 Running 0 11h openshift-etcd etcd-member-ostest-wg9bj-master-0 2/2 Running 0 12h openshift-etcd etcd-member-ostest-wg9bj-master-1 2/2 Running 0 12h openshift-etcd etcd-member-ostest-wg9bj-master-2 2/2 Running 0 12h openshift-insights insights-operator-5568799bf7-wzpv6 1/1 Running 0 12h openshift-kube-apiserver-operator kube-apiserver-operator-dc675f755-h5drq 1/1 Running 2 12h openshift-kube-apiserver installer-2-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-apiserver installer-2-ostest-wg9bj-master-1 0/1 Completed 0 12h openshift-kube-apiserver installer-2-ostest-wg9bj-master-2 0/1 Completed 0 12h openshift-kube-apiserver installer-3-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-apiserver installer-3-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-apiserver installer-3-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-apiserver kube-apiserver-ostest-wg9bj-master-0 3/3 Running 0 11h openshift-kube-apiserver kube-apiserver-ostest-wg9bj-master-1 3/3 Running 0 11h openshift-kube-apiserver kube-apiserver-ostest-wg9bj-master-2 3/3 Running 0 11h openshift-kube-apiserver revision-pruner-2-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-apiserver revision-pruner-2-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-apiserver revision-pruner-2-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-apiserver revision-pruner-3-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-apiserver revision-pruner-3-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-apiserver revision-pruner-3-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-controller-manager-operator kube-controller-manager-operator-586f864f89-crr88 1/1 Running 2 12h openshift-kube-controller-manager installer-2-ostest-wg9bj-master-0 0/1 Completed 0 12h openshift-kube-controller-manager installer-6-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-controller-manager installer-6-ostest-wg9bj-master-1 0/1 Completed 0 12h openshift-kube-controller-manager installer-6-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-controller-manager installer-7-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-controller-manager installer-7-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-controller-manager installer-7-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-controller-manager kube-controller-manager-ostest-wg9bj-master-0 2/2 Running 1 11h openshift-kube-controller-manager kube-controller-manager-ostest-wg9bj-master-1 2/2 Running 0 11h openshift-kube-controller-manager kube-controller-manager-ostest-wg9bj-master-2 2/2 Running 1 11h openshift-kube-controller-manager revision-pruner-2-ostest-wg9bj-master-0 0/1 Completed 0 12h openshift-kube-controller-manager revision-pruner-6-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-controller-manager revision-pruner-6-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-controller-manager revision-pruner-6-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-controller-manager revision-pruner-7-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-controller-manager revision-pruner-7-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-controller-manager revision-pruner-7-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-scheduler-operator openshift-kube-scheduler-operator-644d57c785-wqtxg 1/1 Running 2 12h openshift-kube-scheduler installer-2-ostest-wg9bj-master-2 0/1 Completed 0 12h openshift-kube-scheduler installer-4-ostest-wg9bj-master-0 0/1 Completed 0 12h openshift-kube-scheduler installer-4-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-scheduler installer-4-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kube-scheduler openshift-kube-scheduler-ostest-wg9bj-master-0 1/1 Running 0 12h openshift-kube-scheduler openshift-kube-scheduler-ostest-wg9bj-master-1 1/1 Running 1 11h openshift-kube-scheduler openshift-kube-scheduler-ostest-wg9bj-master-2 1/1 Running 1 11h openshift-kube-scheduler revision-pruner-2-ostest-wg9bj-master-2 0/1 Completed 0 12h openshift-kube-scheduler revision-pruner-4-ostest-wg9bj-master-0 0/1 Completed 0 11h openshift-kube-scheduler revision-pruner-4-ostest-wg9bj-master-1 0/1 Completed 0 11h openshift-kube-scheduler revision-pruner-4-ostest-wg9bj-master-2 0/1 Completed 0 11h openshift-kuryr kuryr-cni-48f6d 1/1 Running 0 12h openshift-kuryr kuryr-cni-55hw8 1/1 Running 0 12h openshift-kuryr kuryr-cni-6swr7 1/1 Running 0 12h openshift-kuryr kuryr-cni-dqdc7 1/1 Running 0 11h openshift-kuryr kuryr-cni-gszvd 1/1 Running 0 11h openshift-kuryr kuryr-cni-z72wc 1/1 Running 0 11h openshift-kuryr kuryr-controller-8557787669-ldqj5 1/1 Running 3 12h openshift-kuryr kuryr-dns-admission-controller-8wx5l 1/1 Running 0 12h openshift-kuryr kuryr-dns-admission-controller-lqdq6 1/1 Running 0 12h openshift-kuryr kuryr-dns-admission-controller-z2zbp 1/1 Running 0 12h openshift-machine-api machine-api-controllers-7b9c567969-rw52l 3/3 Running 0 12h openshift-machine-api machine-api-operator-668d945596-j2xvc 1/1 Running 1 12h openshift-machine-config-operator etcd-quorum-guard-74fbfccd87-2mrgb 1/1 Running 0 12h openshift-machine-config-operator etcd-quorum-guard-74fbfccd87-7qgkk 1/1 Running 0 12h openshift-machine-config-operator etcd-quorum-guard-74fbfccd87-hmtqn 1/1 Running 0 12h openshift-machine-config-operator machine-config-controller-c97d65bd7-9zt4t 1/1 Running 1 12h openshift-machine-config-operator machine-config-daemon-c648v 1/1 Running 0 11h openshift-machine-config-operator machine-config-daemon-hrcn6 1/1 Running 0 12h openshift-machine-config-operator machine-config-daemon-kkgft 1/1 Running 0 11h openshift-machine-config-operator machine-config-daemon-ng22z 1/1 Running 0 11h openshift-machine-config-operator machine-config-daemon-nwjs6 1/1 Running 0 12h openshift-machine-config-operator machine-config-daemon-sdpsp 1/1 Running 0 12h openshift-machine-config-operator machine-config-operator-8467996985-w7skh 1/1 Running 1 12h openshift-machine-config-operator machine-config-server-89d5w 1/1 Running 0 12h openshift-machine-config-operator machine-config-server-cbhhv 1/1 Running 0 12h openshift-machine-config-operator machine-config-server-xjxfx 1/1 Running 0 12h openshift-multus multus-2qfng 1/1 Running 0 11h openshift-multus multus-8zgpl 1/1 Running 0 11h openshift-multus multus-admission-controller-2zbgt 1/1 Running 0 12h openshift-multus multus-admission-controller-94jtk 1/1 Running 0 12h openshift-multus multus-admission-controller-z5pjg 1/1 Running 0 12h openshift-multus multus-h7h27 1/1 Running 0 12h openshift-multus multus-hf87d 1/1 Running 0 12h openshift-multus multus-nc9vz 1/1 Running 0 12h openshift-multus multus-z5sgz 1/1 Running 0 11h openshift-network-operator network-operator-7fdc49495d-44nfj 1/1 Running 0 12h openshift-openstack-infra coredns-ostest-wg9bj-master-0 1/1 Running 0 12h openshift-openstack-infra coredns-ostest-wg9bj-master-1 1/1 Running 0 12h openshift-openstack-infra coredns-ostest-wg9bj-master-2 1/1 Running 0 12h openshift-openstack-infra coredns-ostest-wg9bj-worker-cmnlk 1/1 Running 0 11h openshift-openstack-infra coredns-ostest-wg9bj-worker-r4sj5 1/1 Running 0 11h openshift-openstack-infra coredns-ostest-wg9bj-worker-wdmxw 1/1 Running 0 11h openshift-openstack-infra haproxy-ostest-wg9bj-master-0 2/2 Running 1 12h openshift-openstack-infra haproxy-ostest-wg9bj-master-1 2/2 Running 1 12h openshift-openstack-infra haproxy-ostest-wg9bj-master-2 2/2 Running 1 12h openshift-openstack-infra keepalived-ostest-wg9bj-master-0 1/1 Running 0 12h openshift-openstack-infra keepalived-ostest-wg9bj-master-1 1/1 Running 0 12h openshift-openstack-infra keepalived-ostest-wg9bj-master-2 1/1 Running 0 12h openshift-openstack-infra keepalived-ostest-wg9bj-worker-cmnlk 1/1 Running 0 11h openshift-openstack-infra keepalived-ostest-wg9bj-worker-r4sj5 1/1 Running 0 11h openshift-openstack-infra keepalived-ostest-wg9bj-worker-wdmxw 1/1 Running 0 11h openshift-openstack-infra mdns-publisher-ostest-wg9bj-master-0 1/1 Running 0 12h openshift-openstack-infra mdns-publisher-ostest-wg9bj-master-1 1/1 Running 0 12h openshift-openstack-infra mdns-publisher-ostest-wg9bj-master-2 1/1 Running 0 12h openshift-openstack-infra mdns-publisher-ostest-wg9bj-worker-cmnlk 1/1 Running 0 11h openshift-openstack-infra mdns-publisher-ostest-wg9bj-worker-r4sj5 1/1 Running 0 11h openshift-openstack-infra mdns-publisher-ostest-wg9bj-worker-wdmxw 1/1 Running 0 11h openshift-operator-lifecycle-manager catalog-operator-67d65c7774-hk7pk 1/1 Running 0 12h openshift-operator-lifecycle-manager olm-operator-8559ddbb5c-5dxv2 1/1 Running 0 12h openshift-operator-lifecycle-manager packageserver-b5689448f-wdd4v 1/1 Running 0 11h openshift-operator-lifecycle-manager packageserver-b5689448f-whhzk 1/1 Running 0 11h openshift-service-ca-operator service-ca-operator-8655757d55-jz5h6 1/1 Running 1 12h openshift-service-ca apiservice-cabundle-injector-7c65c88bd9-fpm7x 1/1 Running 1 12h openshift-service-ca configmap-cabundle-injector-6864bf47f7-9pfv2 1/1 Running 1 12h openshift-service-ca service-serving-cert-signer-77c9c666c9-gtbsl 1/1 Running 1 12h (shiftstack) [stack@undercloud-0 ~]$ openstack port list | grep 172.30.0.1 | 0abd15ce-2536-452a-b2b9-054421f506d5 | octavia-lb-dbbae311-14c3-4298-a228-4df5e500275e | fa:16:3e:43:73:11 | ip_address='172.30.0.10', subnet_id='d4c54fd6-9550-4eee-b590-a46abe5a721f' | DOWN | | 57d29893-fbb9-4b2a-93e0-6b07ab5dc8e7 | octavia-lb-9f6f2c5f-4fdd-4986-8d39-1d59768206ef | fa:16:3e:2b:58:a8 | ip_address='172.30.0.1', subnet_id='d4c54fd6-9550-4eee-b590-a46abe5a721f' | DOWN | --- Additional comment from Ben Bennett on 2019-12-10 14:20:50 UTC --- Pushing to 4.4 since this is not a 4.3 regression. When we have a fix for it, we can talk about doing a back-port.
Closing this as the bug was in OpenStack side instead of OCP side. So no need to have one for 4.4 and one for 4.3. The 4.4 bug was already moved to OSP13 (neutron): https://bugzilla.redhat.com/show_bug.cgi?id=1779654 *** This bug has been marked as a duplicate of bug 1779654 ***