Bug 1983153 - WMCO auto upgrade from v2.0.2 to v3.0.0 failed
Summary: WMCO auto upgrade from v2.0.2 to v3.0.0 failed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Windows Containers
Version: 4.8
Hardware: x86_64
OS: Linux
urgent
high
Target Milestone: ---
: 4.8.z
Assignee: Mansi Kulkarni
QA Contact: gaoshang
URL:
Whiteboard:
Depends On: 1982991
Blocks: 1985349
TreeView+ depends on / blocked
 
Reported: 2021-07-16 16:46 UTC by Mansi Kulkarni
Modified: 2021-08-04 05:46 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1982991
: 1985349 (view as bug list)
Environment:
Last Closed: 2021-08-03 20:29:16 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift windows-machine-config-operator pull 528 0 None closed [release-4.8] Bug 1983153: Remove Service account file from bundle manifests 2021-07-23 16:05:50 UTC
Github openshift windows-machine-config-operator pull 530 0 None open [release-4.8] Bug 1983153: Fix e2e log collection and the upgrade process by switching back to more descriptive WMCO lab... 2021-07-27 16:27:21 UTC
Red Hat Product Errata RHSA-2021:3001 0 None None None 2021-08-03 20:29:49 UTC

Description Mansi Kulkarni 2021-07-16 16:46:11 UTC
+++ This bug was initially created as a clone of Bug #1982991 +++

Description of problem:
When upgrading from v2.0.2 to v3.0.0, operator stuck in pending status
$ oc get csv -n openshift-windows-machine-config-operator
NAME                                     DISPLAY                            VERSION    REPLACES                                 PHASE
elasticsearch-operator.5.0.6-35          OpenShift Elasticsearch Operator   5.0.6-35                                            Succeeded
windows-machine-config-operator.v2.0.2   Windows Machine Config Operator    2.0.2      windows-machine-config-operator.v2.0.1   Replacing
windows-machine-config-operator.v3.0.0   Windows Machine Config Operator    3.0.0      windows-machine-config-operator.v2.0.2   Pending

Version-Release number of selected component (if applicable):
OCP upgrade from 4.7.0-0.nightly-2021-07-15-180731 to 4.8.0-0.nightly-2021-07-15-150855

How reproducible:
always

Steps to Reproduce:
1, Install WMCO 2.0.2 on OCP 4.7 with 4.7 stage catalogsource, set "update approve" as automatic
e.g.
$ oc get catalogsource wmco -oyaml -n openshift-marketplace | grep " image"
  image: brew.registry.redhat.io/rh-osbs/iib-pub-pending:v4.7

2, Upgrade OCP 4.7 to 4.8, refer to https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-21588

$ oc adm release info registry.ci.openshift.org/ocp/release:4.8.0-0.nightly-2021-07-15-150855 | grep 'Pull From:'
Pull From: registry.ci.openshift.org/ocp/release@sha256:c3ec3e58ec67160900a5fb8d8ee51b81567861e7bba3f809e45d8be81fe8673b

$ oc adm upgrade --to-image=registry.ci.openshift.org/ocp/release@sha256:c3ec3e58ec67160900a5fb8d8ee51b81567861e7bba3f809e45d8be81fe8673b --allow-explicit-upgrade --force

3, Change WMCO catalogsource to 4.8 stage
e.g.
$ oc get catalogsource wmco -oyaml -n openshift-marketplace | grep " image"
  image: brew.registry.redhat.io/rh-osbs/iib-pub-pending:v4.8

4, Wait and check WMCO auto upgrade

Actual results:
Operator stuck in pending status

Expected results:
Operator should upgrade to 3.0.0

Additional info:
Describe csv, found "RequirementsNotMet, one or more requirements couldn't be found"

$ oc describe csv windows-machine-config-operator.v3.0.0
...
Status:
  Cleanup:
  Conditions:
    Last Transition Time:  2021-07-15T14:30:21Z
    Last Update Time:      2021-07-15T14:30:21Z
    Message:               requirements not yet checked
    Phase:                 Pending
    Reason:                RequirementsUnknown
    Last Transition Time:  2021-07-15T14:30:21Z
    Last Update Time:      2021-07-15T14:30:21Z
    Message:               one or more requirements couldn't be found
    Phase:                 Pending
    Reason:                RequirementsNotMet
  Last Transition Time:    2021-07-15T14:30:21Z
  Last Update Time:        2021-07-15T14:30:21Z
  Message:                 one or more requirements couldn't be found
  Phase:                   Pending
  Reason:                  RequirementsNotMet
  Requirement Status:
    Group:    operators.coreos.com
    Kind:     ClusterServiceVersion
    Message:  CSV minKubeVersion (1.21.0) less than server version (v1.21.1+f36aa36)
    Name:     windows-machine-config-operator.v3.0.0
    Status:   Present
    Version:  v1alpha1
    Group:    
    Kind:     ServiceAccount
    Message:  Service account is owned by another ClusterServiceVersion
    Name:     windows-machine-config-operator
    Status:   PresentNotSatisfied
    Version:  v1
Events:
  Type    Reason               Age                From                        Message
  ----    ------               ----               ----                        -------
  Normal  RequirementsUnknown  33m (x2 over 33m)  operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   33m (x2 over 33m)  operator-lifecycle-manager  one or more requirements couldn't be found

--- Additional comment from Aravindh Puthiyaparambil on 2021-07-16 16:13:26 UTC ---

This is a blocker for the WMCO 3.0.0 release

Comment 2 gaoshang 2021-07-21 07:12:33 UTC
Upgrade to the new WMCO 3.0.0, operator still leaves in pending status, changed back bug status.

Comment 3 Aravindh Puthiyaparambil 2021-07-23 16:14:07 UTC
Marking as VERIFIED to allow the release-4.7 PR to merge. Will move this back to ON_QA once that PR merges.

Comment 5 errata-xmlrpc 2021-08-03 20:29:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Container Platform for Windows Containers 3.0.0 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3001

Comment 6 Ronnie Rasouli 2021-08-04 05:46:27 UTC
Verified
rrasouli@rrasouli-mac openshift-tests-private % oc get csv
NAME                                     DISPLAY                            VERSION   REPLACES                                 PHASE
elasticsearch-operator.5.0.7-9           OpenShift Elasticsearch Operator   5.0.7-9                                            Succeeded
windows-machine-config-operator.v3.0.0   Windows Machine Config Operator    3.0.0     windows-machine-config-operator.v2.0.3   Succeeded
rrasouli@rrasouli-mac openshift-tests-private % oc get catalogsource wmco -oyaml -n openshift-marketplace | grep " image"
  image: brew.registry.redhat.io/rh-osbs/iib-pub-pending:v4.8


Note You need to log in before you can comment on or make changes to this bug.