Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1731323

Summary: Some operators cannot be upgraded from 4.1 to 4.2
Product: OpenShift Container Platform Reporter: Wenjing Zheng <wzheng>
Component: NetworkingAssignee: Miciah Dashiel Butler Masters <mmasters>
Networking sub component: router QA Contact: Hongan Li <hongli>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: aos-bugs, cdc, dmace, gpei, hongli, jokerman, mmccomas, yapei
Version: 4.2.0Keywords: TestBlocker
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1734192 1734414 (view as bug list) Environment:
Last Closed: 2019-10-16 06:30:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Log from cluster-version-operator none

Description Wenjing Zheng 2019-07-19 06:11:37 UTC
Created attachment 1591904 [details]
Log from cluster-version-operator

Description of problem:
dns, network and machine-config cannot be upgraded from 4.1.6 to 4.2 nightly:
$ oc get clusteroperator
NAME                                       VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
cloud-credential                           4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
cluster-autoscaler                         4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
console                                    4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
dns                                        4.1.6                               True        False         False      21h
image-registry                             4.2.0-0.nightly-2019-07-18-032637   True        False         False      20h
ingress                                    4.2.0-0.nightly-2019-07-15-231921   True        False         False      21h
kube-apiserver                             4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
kube-controller-manager                    4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
kube-scheduler                             4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
machine-api                                4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
machine-config                             4.1.6                               True        False         False      21h
marketplace                                4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h
monitoring                                 4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h
network                                    4.1.6                               True        False         False      21h
node-tuning                                4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h
openshift-apiserver                        4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h
openshift-controller-manager               4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
openshift-samples                          4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h
operator-lifecycle-manager                 4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
operator-lifecycle-manager-catalog         4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
operator-lifecycle-manager-packageserver                                       False       False         False      19h
service-ca                                 4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
service-catalog-apiserver                  4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
service-catalog-controller-manager         4.2.0-0.nightly-2019-07-18-032637   True        False         False      21h
storage                                    4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h
support                                    4.2.0-0.nightly-2019-07-18-032637   True        False         False      19h


Version-Release number of selected component (if applicable):
4.1.4->4.1.6->4.2 nightly build

How reproducible:
Always

Steps to Reproduce:
1.Prepare a 4.1 cluster
2.oc adm upgrade --to-image=registry.svc.ci.openshift.org/ocp/release:4.2.0-0.nightly-2019-07-18-032637 --force=true
3.

Actual results:
dns, network and machine-config remains 4.1 version

Expected results:
They should be upgrade like other Operators.

Additional info:
CVO log is in attachment.

Comment 1 Abhinav Dahiya 2019-07-19 16:40:36 UTC
as apparent from the logs, CVO is waiting for ingress-operator deployment to roll-out. it cannot proceed without that.

can you include `oc adm must-gather` to debug why?

Comment 5 Yadan Pei 2019-07-29 08:31:13 UTC
CVO logs shows:
oc logs -f console-operator-756fbfcf59-dq7z7 -n openshift-console-operator
....
W0729 08:07:19.790348       1 reflector.go:289] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.
W0729 08:08:20.738596       1 reflector.go:289] github.com/openshift/client-go/oauth/informers/externalversions/factory.go:101: watch of *v1.OAuthClient ended with: The resourceVersion for the provided watch is too old.
....

Comment 8 Hongan Li 2019-07-31 03:22:17 UTC
Seems the root cause is ingress operator and have opened two separate BUGs to track them:

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1734192
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1734414

Since the first one has been verified on AWS platform so I'm closing this one.

Comment 9 errata-xmlrpc 2019-10-16 06:30:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922