Bug 2213451

Summary: Set ​​maxOpenShiftVersion to block OpenShift that didn't upgrade ODF version
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Eran Tamir <etamir>
Component: odf-operatorAssignee: Nitin Goyal <nigoyal>
Status: CLOSED ERRATA QA Contact: Vijay Avuthu <vavuthu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.11CC: bkunal, ebenahar, kramdoss, muagarwa, nigoyal, ocs-bugs, odf-bz-bot, vavuthu
Target Milestone: ---   
Target Release: ODF 4.11.9   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.11.9-2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2213450
: 2213452 (view as bug list) Environment:
Last Closed: 2023-07-20 16:12:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2213452    
Bug Blocks: 2213450    

Description Eran Tamir 2023-06-08 07:59:18 UTC 
Description of problem (please be detailed as possible and provide log
snippests):

Currently, we don't block OpenShift upgrades when the ODF version is not compatible. For example, when the ODF version is 4.11 and the OpenShift version is 4.12, we should block an upgrade to 4.13 as ODF 4.11 is incompatible. 



Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

Yes. It may cause and issue and ends with a support call. 

Is there any workaround available to the best of your knowledge?

Sometimes manual changes in the CSV can help, but it involves support and may help just to upgrade to the relevant ODF version, while part of the service may be impacted. 

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?

1

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?

Yes

--- Additional comment from RHEL Program Management on 2023-06-08 07:57:05 UTC ---

This bug having no release flag set previously, is now set with release flag 'odf‑4.13.0' to '?', and so is being proposed to be fixed at the ODF 4.13.0 release. Note that the 3 Acks (pm_ack, devel_ack, qa_ack), if any previously set while release flag was missing, have now been reset since the Acks are to be set against a release flag.
Comment 5 Nitin Goyal 2023-07-03 06:11:56 UTC 
PR is merged.
Comment 8 Vijay Avuthu 2023-07-18 06:10:51 UTC 
Update:
=========
1. Installed OCP 4.11 ( 4.11.44 ) + ODF 4.11 ( ocs-registry:4.11.9-2 ) ( job:  https://url.corp.redhat.com/24fc7d1 )
2. chnaged channel to stable-4.12 and upgraded to latest images in channel
$ oc adm upgrade --to-latest
Requesting update to 4.12.23

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.12.23   True        False         8s      Cluster version is 4.12.23


3. set channel to candidate-4.13 and upgraded to latest image available

$ oc adm upgrade
Cluster version is 4.12.23

Upgradeable=False

  Reason: IncompatibleOperatorsInstalled
  Message: Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12

ReleaseAccepted=False

  Reason: PreconditionChecks
  Message: Preconditions failed for payload loaded version="4.13.5" image="registry.ci.openshift.org/ocp/release@sha256:af19e94813478382e36ae1fa2ae7bbbff1f903dded6180f4eb0624afe6fc6cd4": Multiple precondition checks failed:
  * Precondition "ClusterVersionUpgradeable" failed because of "IncompatibleOperatorsInstalled": Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12
  * Precondition "ClusterVersionRecommendedUpdate" failed because of "UnknownUpdate": RetrievedUpdates=True (), so the update from 4.12.23 to 4.13.5 is probably neither recommended nor supported.

Upstream is unset, so the cluster will use an appropriate default.
Channel: candidate-4.13 (available channels: candidate-4.12, candidate-4.13, eus-4.12, fast-4.12, stable-4.12)

Recommended updates:

  VERSION     IMAGE
  4.12.24     quay.io/openshift-release-dev/ocp-release@sha256:b0b11eedf91175459b5d7aefcf3936d0cabf00f01ced756677483f5f26227328

$ oc adm upgrade --to-latest
Requesting update to 4.12.24

4. again upgrade to 4.12.25 whch is latest in channel

$ oc adm upgrade
Cluster version is 4.12.24

Upgradeable=False

  Reason: IncompatibleOperatorsInstalled
  Message: Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12

Upstream is unset, so the cluster will use an appropriate default.
Channel: candidate-4.13 (available channels: candidate-4.12, candidate-4.13, fast-4.12)

Recommended updates:

  VERSION     IMAGE
  4.12.25     quay.io/openshift-release-dev/ocp-release@sha256:5a4fb052cda1d14d1e306ce87e6b0ded84edddaa76f1cf401bcded99cef2ad84
$ oc adm upgrade --to-latest
Requesting update to 4.12.25
$

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.12.25   True        False         13h     Cluster version is 4.12.25


5. 4.13.5 is available in channel and tried to upgrade

$ oc adm upgrade --to-latest
Requesting update to 4.13.5

As expected , upgrade is not started

$ oc adm upgrade
Cluster version is 4.12.25

Upgradeable=False

  Reason: IncompatibleOperatorsInstalled
  Message: Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12

ReleaseAccepted=False

  Reason: PreconditionChecks
  Message: Preconditions failed for payload loaded version="4.13.5" image="quay.io/openshift-release-dev/ocp-release@sha256:af19e94813478382e36ae1fa2ae7bbbff1f903dded6180f4eb0624afe6fc6cd4": Precondition "ClusterVersionUpgradeable" failed because of "IncompatibleOperatorsInstalled": Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12

Upstream is unset, so the cluster will use an appropriate default.
Channel: candidate-4.13 (available channels: candidate-4.12, candidate-4.13)

Recommended updates:

  VERSION     IMAGE
  4.13.5      quay.io/openshift-release-dev/ocp-release@sha256:af19e94813478382e36ae1fa2ae7bbbff1f903dded6180f4eb0624afe6fc6cd4

> tried upgrading using digest as well and upgrade is not triggered as expected

$ oc adm upgrade --to-image=registry.ci.openshift.org/ocp/release@sha256:af19e94813478382e36ae1fa2ae7bbbff1f903dded6180f4eb0624afe6fc6cd4 --allow-explicit-upgrade
Requesting update to 4.13.5
$

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.12.25   True        False         13h     Cluster version is 4.12.25

$ oc adm upgrade
Cluster version is 4.12.25

Upgradeable=False

  Reason: IncompatibleOperatorsInstalled
  Message: Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12

ReleaseAccepted=False

  Reason: PreconditionChecks
  Message: Preconditions failed for payload loaded version="4.13.5" image="quay.io/openshift-release-dev/ocp-release@sha256:af19e94813478382e36ae1fa2ae7bbbff1f903dded6180f4eb0624afe6fc6cd4": Precondition "ClusterVersionUpgradeable" failed because of "IncompatibleOperatorsInstalled": Cluster operator operator-lifecycle-manager should not be upgraded between minor versions: ClusterServiceVersions blocking cluster upgrade: openshift-storage/odf-operator.v4.11.9 is incompatible with OpenShift minor versions greater than 4.12

Upstream is unset, so the cluster will use an appropriate default.
Channel: candidate-4.13 (available channels: candidate-4.12, candidate-4.13)

Recommended updates:

  VERSION     IMAGE
  4.13.5      quay.io/openshift-release-dev/ocp-release@sha256:af19e94813478382e36ae1fa2ae7bbbff1f903dded6180f4eb0624afe6fc6cd4
$ 

> checked olm.properties and maxOpenShiftVersion version is set properly

$ oc get csv odf-operator.v4.11.9 -o json | jq '.metadata | .annotations | ."olm.properties"'
"[{\"type\": \"olm.maxOpenShiftVersion\", \"value\": \"4.12\"}]"
$
Comment 13 errata-xmlrpc 2023-07-20 16:12:43 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Data Foundation 4.11.9 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:4238