Bug 1982991 - WMCO auto upgrade from v2.0.2 to v3.0.0 failed
Summary: WMCO auto upgrade from v2.0.2 to v3.0.0 failed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Windows Containers
Version: 4.9
Hardware: x86_64
OS: Linux
urgent
high
Target Milestone: ---
: 4.9.0
Assignee: Mansi Kulkarni
QA Contact: gaoshang
URL:
Whiteboard:
Depends On:
Blocks: 1983153
TreeView+ depends on / blocked
 
Reported: 2021-07-16 08:23 UTC by gaoshang
Modified: 2021-10-28 17:41 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1983153 (view as bug list)
Environment:
Last Closed: 2021-10-28 17:41:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift windows-machine-config-operator pull 520 0 None closed Bug 1982991: Remove Service account file from bundle manifests 2021-07-23 16:08:49 UTC
Red Hat Product Errata RHBA-2021:3702 0 None None None 2021-10-28 17:41:40 UTC

Description gaoshang 2021-07-16 08:23:52 UTC
Description of problem:
When upgrading from v2.0.2 to v3.0.0, operator stuck in pending status
$ oc get csv -n openshift-windows-machine-config-operator
NAME                                     DISPLAY                            VERSION    REPLACES                                 PHASE
elasticsearch-operator.5.0.6-35          OpenShift Elasticsearch Operator   5.0.6-35                                            Succeeded
windows-machine-config-operator.v2.0.2   Windows Machine Config Operator    2.0.2      windows-machine-config-operator.v2.0.1   Replacing
windows-machine-config-operator.v3.0.0   Windows Machine Config Operator    3.0.0      windows-machine-config-operator.v2.0.2   Pending

Version-Release number of selected component (if applicable):
OCP upgrade from 4.7.0-0.nightly-2021-07-15-180731 to 4.8.0-0.nightly-2021-07-15-150855

How reproducible:
always

Steps to Reproduce:
1, Install WMCO 2.0.2 on OCP 4.7 with 4.7 stage catalogsource, set "update approve" as automatic
e.g.
$ oc get catalogsource wmco -oyaml -n openshift-marketplace | grep " image"
  image: brew.registry.redhat.io/rh-osbs/iib-pub-pending:v4.7

2, Upgrade OCP 4.7 to 4.8, refer to https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-21588

$ oc adm release info registry.ci.openshift.org/ocp/release:4.8.0-0.nightly-2021-07-15-150855 | grep 'Pull From:'
Pull From: registry.ci.openshift.org/ocp/release@sha256:c3ec3e58ec67160900a5fb8d8ee51b81567861e7bba3f809e45d8be81fe8673b

$ oc adm upgrade --to-image=registry.ci.openshift.org/ocp/release@sha256:c3ec3e58ec67160900a5fb8d8ee51b81567861e7bba3f809e45d8be81fe8673b --allow-explicit-upgrade --force

3, Change WMCO catalogsource to 4.8 stage
e.g.
$ oc get catalogsource wmco -oyaml -n openshift-marketplace | grep " image"
  image: brew.registry.redhat.io/rh-osbs/iib-pub-pending:v4.8

4, Wait and check WMCO auto upgrade

Actual results:
Operator stuck in pending status

Expected results:
Operator should upgrade to 3.0.0

Additional info:
Describe csv, found "RequirementsNotMet, one or more requirements couldn't be found"

$ oc describe csv windows-machine-config-operator.v3.0.0
...
Status:
  Cleanup:
  Conditions:
    Last Transition Time:  2021-07-15T14:30:21Z
    Last Update Time:      2021-07-15T14:30:21Z
    Message:               requirements not yet checked
    Phase:                 Pending
    Reason:                RequirementsUnknown
    Last Transition Time:  2021-07-15T14:30:21Z
    Last Update Time:      2021-07-15T14:30:21Z
    Message:               one or more requirements couldn't be found
    Phase:                 Pending
    Reason:                RequirementsNotMet
  Last Transition Time:    2021-07-15T14:30:21Z
  Last Update Time:        2021-07-15T14:30:21Z
  Message:                 one or more requirements couldn't be found
  Phase:                   Pending
  Reason:                  RequirementsNotMet
  Requirement Status:
    Group:    operators.coreos.com
    Kind:     ClusterServiceVersion
    Message:  CSV minKubeVersion (1.21.0) less than server version (v1.21.1+f36aa36)
    Name:     windows-machine-config-operator.v3.0.0
    Status:   Present
    Version:  v1alpha1
    Group:    
    Kind:     ServiceAccount
    Message:  Service account is owned by another ClusterServiceVersion
    Name:     windows-machine-config-operator
    Status:   PresentNotSatisfied
    Version:  v1
Events:
  Type    Reason               Age                From                        Message
  ----    ------               ----               ----                        -------
  Normal  RequirementsUnknown  33m (x2 over 33m)  operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   33m (x2 over 33m)  operator-lifecycle-manager  one or more requirements couldn't be found

Comment 1 Aravindh Puthiyaparambil 2021-07-16 16:13:26 UTC
This is a blocker for the WMCO 3.0.0 release

Comment 2 gaoshang 2021-07-21 07:10:38 UTC
Upgrade to the new WMCO 3.0.0, operator still leaves in pending status, changed back bug status.

Comment 3 Aravindh Puthiyaparambil 2021-07-23 16:14:27 UTC
Marking as VERIFIED to allow the release-4.7 PR to merge. Will move this back to ON_QA once that PR merges.

Comment 4 Aravindh Puthiyaparambil 2021-07-27 16:26:44 UTC
Marking as VERIFIED to allow the release-4.8 PR to merge. Will move this back to ON_QA once that PR merges.

Comment 5 Ronnie Rasouli 2021-08-01 10:04:05 UTC
rrasouli@rrasouli-mac openshift-tests-private % oc get csv
NAME                                     DISPLAY                            VERSION   REPLACES                                 PHASE
elasticsearch-operator.5.0.7-9           OpenShift Elasticsearch Operator   5.0.7-9                                            Succeeded
windows-machine-config-operator.v3.0.0   Windows Machine Config Operator    3.0.0     windows-machine-config-operator.v2.0.3   Succeeded
rrasouli@rrasouli-mac openshift-tests-private % oc get catalogsource wmco -oyaml -n openshift-marketplace | grep " image"
  image: brew.registry.redhat.io/rh-osbs/iib-pub-pending:v4.8

Comment 7 Yu Li 2021-08-05 02:48:15 UTC
As discussed with Mansi and Aravindh, this bug is mainly used to trace code merge which was be fixed on master branch. The issue was happening on upgrade from WMCO 2.0.2 to WMCO 3.0.0. WMCO 2.0.x is running on OCP4.7. WMCO 3.0.x is running on OCP4.8. OCP4.9 is not in the picture right now. When have WMCO 4.y.z (for OCP4.9), QE will run upgrade test from WMCO 3.y.z to 4.y.z, then we will see whether we have new issues and open new bug based on that only.

Comment 12 errata-xmlrpc 2021-10-28 17:41:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Windows Container Support for Red Hat OpenShift 4.0.0 product release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3702


Note You need to log in before you can comment on or make changes to this bug.