Description of problem: Upgrade from 4.2 to 4.3 never ends on Openstack as networking is not finishing the upgrade $ oc get all -n openshift-network-operator NAME READY STATUS RESTARTS AGE pod/network-operator-7f54fd5c9d-jp76x 0/1 CrashLoopBackOff 6 11m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/network-operator 0/1 1 0 3h37m NAME DESIRED CURRENT READY AGE replicaset.apps/network-operator-7f54fd5c9d 1 1 0 11m replicaset.apps/network-operator-c66bcc9d5 0 0 0 3h37m $ oc logs pod/network-operator-7f54fd5c9d-jp76x 2019/10/23 14:47:15 Go Version: go1.12.8 2019/10/23 14:47:15 Go OS/Arch: linux/amd64 2019/10/23 14:47:15 operator-sdk Version: v0.11.0 2019/10/23 14:47:15 overriding kubernetes api to https://api-int.mrnd-13-42-no.qe.devcluster.openshift.com:6443 2019/10/23 14:47:19 error listening on :8080: listen tcp :8080: bind: address already in use Version-Release number of the following components: rpm -q openshift-ansible rpm -q ansible ansible --version How reproducible: Steps to Reproduce: 1.Install any version of 4.2 on Openstack 2.Update to 4.3 (oc adm upgrade --to-image=registry.svc.ci.openshift.org/ocp/release:4.3.0-0.nightly-2019-10-23-140012 --force) 3.Check status of all operators Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
Ricky, can you take a look? Did the update to controller-runtime do this? Perhaps it opened a metrics port.
Yep, looks like it: https://github.com/kubernetes-sigs/controller-runtime/commit/410863e4e98487076f7d70d153400fa7f8cba56c
https://github.com/openshift/cluster-network-operator/pull/365
Verified this bug, upgrade from 4.2 to 4.3, see below. history: - completionTime: "2019-10-29T03:47:33Z" image: registry.svc.ci.openshift.org/ocp/release:4.3.0-0.nightly-2019-10-28-232741 startedTime: "2019-10-29T03:13:16Z" state: Completed verified: false version: 4.3.0-0.nightly-2019-10-28-232741 - completionTime: "2019-10-29T02:48:46Z" image: registry.svc.ci.openshift.org/ocp/release@sha256:918db67672a03d16c086fdb54b8a4a77d44b112e558b9a156392964086061640 startedTime: "2019-10-29T02:26:39Z" state: Completed verified: false version: 4.2.0-0.nightly-2019-10-28-140411 observedGeneration: 2 versionHash: hLdX-2pT3K4= kind: List metadata: resourceVersion: "" selfLink: "" [root@dhcp-140-66 ~]# oc get co NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.3.0-0.nightly-2019-10-28-232741 True False False 3h10m cloud-credential 4.3.0-0.nightly-2019-10-28-232741 True False False 3h29m cluster-autoscaler 4.3.0-0.nightly-2019-10-28-232741 True False False 3h25m console 4.3.0-0.nightly-2019-10-28-232741 True False False 131m dns 4.3.0-0.nightly-2019-10-28-232741 True False False 3h29m image-registry 4.3.0-0.nightly-2019-10-28-232741 True False False 140m ingress 4.3.0-0.nightly-2019-10-28-232741 True False False 3h21m insights 4.3.0-0.nightly-2019-10-28-232741 True False False 3h29m kube-apiserver 4.3.0-0.nightly-2019-10-28-232741 True False False 3h27m kube-controller-manager 4.3.0-0.nightly-2019-10-28-232741 True False False 3h27m kube-scheduler 4.3.0-0.nightly-2019-10-28-232741 True False False 3h26m machine-api 4.3.0-0.nightly-2019-10-28-232741 True False False 3h29m machine-config 4.3.0-0.nightly-2019-10-28-232741 True False False 3h28m marketplace 4.3.0-0.nightly-2019-10-28-232741 True False False 138m monitoring 4.3.0-0.nightly-2019-10-28-232741 True False False 130m network 4.3.0-0.nightly-2019-10-28-232741 True False False 3h28m node-tuning 4.3.0-0.nightly-2019-10-28-232741 True False False 134m openshift-apiserver 4.3.0-0.nightly-2019-10-28-232741 True False False 131m openshift-controller-manager 4.3.0-0.nightly-2019-10-28-232741 True False False 3h27m openshift-samples 4.3.0-0.nightly-2019-10-28-232741 True False False 156m operator-lifecycle-manager 4.3.0-0.nightly-2019-10-28-232741 True False False 3h28m operator-lifecycle-manager-catalog 4.3.0-0.nightly-2019-10-28-232741 True False False 3h28m operator-lifecycle-manager-packageserver 4.3.0-0.nightly-2019-10-28-232741 True False False 134m service-ca 4.3.0-0.nightly-2019-10-28-232741 True False False 3h29m service-catalog-apiserver 4.3.0-0.nightly-2019-10-28-232741 True False False 3h25m service-catalog-controller-manager 4.3.0-0.nightly-2019-10-28-232741 True False False 3h25m storage 4.3.0-0.nightly-2019-10-28-232741 True False False 155m
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0107