Bug 1764672 - [OSP] Cannot upgrade from 4.2 to 4.3 because of network-operator port conflict
Summary: [OSP] Cannot upgrade from 4.2 to 4.3 because of network-operator port conflict
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.2.z
Assignee: Ricardo Carrillo Cruz
QA Contact: zhaozhanqi
URL:
Whiteboard:
Depends On: 1769331
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-10-23 14:54 UTC by David Sanz
Modified: 2020-01-22 10:47 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-22 10:46:40 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-network-operator pull 365 0 'None' 'closed' 'Bug 1764672: Disable metrics on controller' 2019-11-19 09:42:17 UTC
Red Hat Product Errata RHBA-2020:0107 0 None None None 2020-01-22 10:47:03 UTC

Description David Sanz 2019-10-23 14:54:06 UTC
Description of problem:

Upgrade from 4.2 to 4.3 never ends on Openstack as networking is not finishing the upgrade

$ oc get all -n openshift-network-operator
NAME                                    READY   STATUS             RESTARTS   AGE
pod/network-operator-7f54fd5c9d-jp76x   0/1     CrashLoopBackOff   6          11m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/network-operator   0/1     1            0           3h37m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/network-operator-7f54fd5c9d   1         1         0       11m
replicaset.apps/network-operator-c66bcc9d5    0         0         0       3h37m

$ oc logs pod/network-operator-7f54fd5c9d-jp76x
2019/10/23 14:47:15 Go Version: go1.12.8
2019/10/23 14:47:15 Go OS/Arch: linux/amd64
2019/10/23 14:47:15 operator-sdk Version: v0.11.0
2019/10/23 14:47:15 overriding kubernetes api to https://api-int.mrnd-13-42-no.qe.devcluster.openshift.com:6443
2019/10/23 14:47:19 error listening on :8080: listen tcp :8080: bind: address already in use



Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.Install any version of 4.2 on Openstack
2.Update to 4.3 (oc adm upgrade --to-image=registry.svc.ci.openshift.org/ocp/release:4.3.0-0.nightly-2019-10-23-140012 --force)
3.Check status of all operators

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Casey Callendrello 2019-10-25 13:08:04 UTC
Ricky, can you take a look? Did the update to controller-runtime do this? Perhaps it opened a metrics port.

Comment 3 Ricardo Carrillo Cruz 2019-10-25 19:12:41 UTC
https://github.com/openshift/cluster-network-operator/pull/365

Comment 5 zhaozhanqi 2019-10-29 05:58:06 UTC
Verified this bug, upgrade from 4.2 to 4.3, see below.

    history:
    - completionTime: "2019-10-29T03:47:33Z"
      image: registry.svc.ci.openshift.org/ocp/release:4.3.0-0.nightly-2019-10-28-232741
      startedTime: "2019-10-29T03:13:16Z"
      state: Completed
      verified: false
      version: 4.3.0-0.nightly-2019-10-28-232741
    - completionTime: "2019-10-29T02:48:46Z"
      image: registry.svc.ci.openshift.org/ocp/release@sha256:918db67672a03d16c086fdb54b8a4a77d44b112e558b9a156392964086061640
      startedTime: "2019-10-29T02:26:39Z"
      state: Completed
      verified: false
      version: 4.2.0-0.nightly-2019-10-28-140411
    observedGeneration: 2
    versionHash: hLdX-2pT3K4=
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""
[root@dhcp-140-66 ~]# oc get co
NAME                                       VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h10m
cloud-credential                           4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h29m
cluster-autoscaler                         4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h25m
console                                    4.3.0-0.nightly-2019-10-28-232741   True        False         False      131m
dns                                        4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h29m
image-registry                             4.3.0-0.nightly-2019-10-28-232741   True        False         False      140m
ingress                                    4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h21m
insights                                   4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h29m
kube-apiserver                             4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h27m
kube-controller-manager                    4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h27m
kube-scheduler                             4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h26m
machine-api                                4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h29m
machine-config                             4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h28m
marketplace                                4.3.0-0.nightly-2019-10-28-232741   True        False         False      138m
monitoring                                 4.3.0-0.nightly-2019-10-28-232741   True        False         False      130m
network                                    4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h28m
node-tuning                                4.3.0-0.nightly-2019-10-28-232741   True        False         False      134m
openshift-apiserver                        4.3.0-0.nightly-2019-10-28-232741   True        False         False      131m
openshift-controller-manager               4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h27m
openshift-samples                          4.3.0-0.nightly-2019-10-28-232741   True        False         False      156m
operator-lifecycle-manager                 4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h28m
operator-lifecycle-manager-catalog         4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h28m
operator-lifecycle-manager-packageserver   4.3.0-0.nightly-2019-10-28-232741   True        False         False      134m
service-ca                                 4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h29m
service-catalog-apiserver                  4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h25m
service-catalog-controller-manager         4.3.0-0.nightly-2019-10-28-232741   True        False         False      3h25m
storage                                    4.3.0-0.nightly-2019-10-28-232741   True        False         False      155m

Comment 8 errata-xmlrpc 2020-01-22 10:46:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0107


Note You need to log in before you can comment on or make changes to this bug.