Bug 1995056

Summary: OCS_CSV_NAME is not correct in odf-operator-manager-config configmap
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Jilju Joy <jijoy>
Component: buildAssignee: Boris Ranto <branto>
Status: CLOSED ERRATA QA Contact: Jilju Joy <jijoy>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.9CC: bniver, ebenahar, kramdoss, madam, muagarwa, nigoyal, ocs-bugs, odf-bz-bot, sostapov
Target Milestone: ---Keywords: TestBlocker
Target Release: ODF 4.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: odf-operator.v4.9.0-102.ci Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-12-13 17:45:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
ClusterServiceVersion not found error none

Description Jilju Joy 2021-08-18 11:26:49 UTC
Created attachment 1815153 [details]
ClusterServiceVersion not found error

Description of problem (please be detailed as possible and provide log
snippests):
Creation of a storage system using odf-operator is failing due to incorrect value of 'OCS_CSV_NAME' parameter in 'odf-operator-manager-config' configmap

$ oc get configmap odf-operator-manager-config -o yaml
apiVersion: v1
data:
  IBM_SUBSCRIPTION_CATALOGSOURCE: odf-catalogsource
  IBM_SUBSCRIPTION_CATALOGSOURCE_NAMESPACE: openshift-marketplace
  IBM_SUBSCRIPTION_CHANNEL: stable-v1
  IBM_SUBSCRIPTION_NAME: ibm-storage-odf-operator
  IBM_SUBSCRIPTION_PACKAGE: ibm-storage-odf-operator
  IBM_SUBSCRIPTION_STARTINGCSV: ibm-storage-odf-operator.v0.2.0
  OCS_CSV_NAME: ocs-operator.v4.9.0
  controller_manager_config.yaml: |
    apiVersion: controller-runtime.sigs.k8s.io/v1alpha1
    kind: ControllerManagerConfig
    health:
      healthProbeBindAddress: :8081
    metrics:
      bindAddress: 127.0.0.1:8080
    webhook:
      port: 9443
    leaderElection:
      leaderElect: true
      resourceName: 4fd470de.openshift.io
kind: ConfigMap
metadata:
  creationTimestamp: "2021-08-18T08:18:46Z"
  labels:
    operators.coreos.com/odf-operator.openshift-storage: ""
  name: odf-operator-manager-config
  namespace: openshift-storage
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: ClusterServiceVersion
    name: odf-operator.v4.9.0-96.ci
    uid: 9013e3a2-ae70-4baf-985b-cabfaa92e5ec
  resourceVersion: "44588"
  uid: b83a5442-a92e-4f3f-805c-bbe71fbfdc72


OCS_CSV_NAME should be 'ocs-operator.v4.9.0-96.ci' instead of 'ocs-operator.v4.9.0'

Error event generated from StorageSystem controller: 
"ClusterServiceVersion.operators.coreos.com "ocs-operator.v4.9.0" not found"

Tested in VMware platform.

CatalogSource yaml:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  labels:
    ocs-operator-internal: 'true'
  name: odf-catalogsource
  namespace: openshift-marketplace
spec:
  displayName: OpenShift Data Foundation
  icon:
    base64data: PHN2ZyBpZD0iTGF5ZXJfMSIgZGF0YS1uYW1lPSJMYXllciAxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAxOTIgMTQ1Ij48ZGVmcz48c3R5bGU+LmNscy0xe2ZpbGw6I2UwMDt9PC9zdHlsZT48L2RlZnM+PHRpdGxlPlJlZEhhdC1Mb2dvLUhhdC1Db2xvcjwvdGl0bGU+PHBhdGggZD0iTTE1Ny43Nyw2Mi42MWExNCwxNCwwLDAsMSwuMzEsMy40MmMwLDE0Ljg4LTE4LjEsMTcuNDYtMzAuNjEsMTcuNDZDNzguODMsODMuNDksNDIuNTMsNTMuMjYsNDIuNTMsNDRhNi40Myw2LjQzLDAsMCwxLC4yMi0xLjk0bC0zLjY2LDkuMDZhMTguNDUsMTguNDUsMCwwLDAtMS41MSw3LjMzYzAsMTguMTEsNDEsNDUuNDgsODcuNzQsNDUuNDgsMjAuNjksMCwzNi40My03Ljc2LDM2LjQzLTIxLjc3LDAtMS4wOCwwLTEuOTQtMS43My0xMC4xM1oiLz48cGF0aCBjbGFzcz0iY2xzLTEiIGQ9Ik0xMjcuNDcsODMuNDljMTIuNTEsMCwzMC42MS0yLjU4LDMwLjYxLTE3LjQ2YTE0LDE0LDAsMCwwLS4zMS0zLjQybC03LjQ1LTMyLjM2Yy0xLjcyLTcuMTItMy4yMy0xMC4zNS0xNS43My0xNi42QzEyNC44OSw4LjY5LDEwMy43Ni41LDk3LjUxLjUsOTEuNjkuNSw5MCw4LDgzLjA2LDhjLTYuNjgsMC0xMS42NC01LjYtMTcuODktNS42LTYsMC05LjkxLDQuMDktMTIuOTMsMTIuNSwwLDAtOC40MSwyMy43Mi05LjQ5LDI3LjE2QTYuNDMsNi40MywwLDAsMCw0Mi41Myw0NGMwLDkuMjIsMzYuMywzOS40NSw4NC45NCwzOS40NU0xNjAsNzIuMDdjMS43Myw4LjE5LDEuNzMsOS4wNSwxLjczLDEwLjEzLDAsMTQtMTUuNzQsMjEuNzctMzYuNDMsMjEuNzdDNzguNTQsMTA0LDM3LjU4LDc2LjYsMzcuNTgsNTguNDlhMTguNDUsMTguNDUsMCwwLDEsMS41MS03LjMzQzIyLjI3LDUyLC41LDU1LC41LDc0LjIyYzAsMzEuNDgsNzQuNTksNzAuMjgsMTMzLjY1LDcwLjI4LDQ1LjI4LDAsNTYuNy0yMC40OCw1Ni43LTM2LjY1LDAtMTIuNzItMTEtMjcuMTYtMzAuODMtMzUuNzgiLz48L3N2Zz4=
    mediatype: image/svg+xml
  image: quay.io/rhceph-dev/ocs-registry:4.9.0-96.ci
  priority: 100
  publisher: Red Hat
  sourceType: grpc
  updateStrategy:
    registryPoll:
      interval: 15m


=================================================================
Version of all relevant components (if applicable):
ocs-operator.v4.9.0-96.ci
odf-operator.v4.9.0-96.ci
OCP 4.9.0-0.nightly-2021-08-17-183417

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes, storage system will not be created.

Is there any workaround available to the best of your knowledge?
Tried editing the the value of OCS_CSV_NAME and restarted odf-operator-controller-manager pod. The initial error was gone.

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. In UI, go to Operators --> Installed Operators and select Openshift Data Foundation.
2. Go to Storage System tab and follow the process to create a storage system.
3. Check the status of storagesystem 


Actual results:
$ oc get storagesystem
NAME                               STORAGE-SYSTEM-KIND                  STORAGE-SYSTEM-NAME   PHASE
odf-storage-system                 storagecluster.ocs.openshift.io/v1   odf-storage-system    Progressing
odf-storage-system-storagesystem   storagecluster.ocs.openshift.io/v1   odf-storage-system    Progressing


(Two storagesystems are present due to the bug #1995043 . )


The storage system will remain in Progressing state. 

Expected results:
Storagesystem should be created. 


Additional info:
Screenshot is attached

Comment 8 Jilju Joy 2021-08-23 11:03:32 UTC
Verified in version:

$ oc get csv
NAME                         DISPLAY                       VERSION        REPLACES   PHASE
ocs-operator.v4.9.0-105.ci   OpenShift Container Storage   4.9.0-105.ci              Succeeded
odf-operator.v4.9.0-105.ci   OpenShift Data Foundation     4.9.0-105.ci              Succeeded


$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.9.0-0.nightly-2021-08-22-070405   True        False         139m    Cluster version is 4.9.0-0.nightly-2021-08-22-070405

Storagesystem and storagecluster got created successfully. 

$ oc get storagesystem
NAME                               STORAGE-SYSTEM-KIND                  STORAGE-SYSTEM-NAME   PHASE
ocs-storagecluster-storagesystem   storagecluster.ocs.openshift.io/v1   ocs-storagecluster    Ready

$ oc get storagecluster
NAME                 AGE   PHASE   EXTERNAL   CREATED AT             VERSION
ocs-storagecluster   31m   Ready              2021-08-23T10:25:46Z   4.9.0


$ oc get configmap odf-operator-manager-config -o yaml | grep OCS_CSV_NAME
  OCS_CSV_NAME: ocs-operator.v4.9.0-105.ci

Installation was done from UI following the steps given in comment #c0

Comment 14 errata-xmlrpc 2021-12-13 17:45:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Data Foundation 4.9.0 enhancement, security, and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:5086