Description of problem (please be detailed as possible and provide log snippests): I have noticed in the engineering upgrade pipeline that it's failing here: https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/889/ I see it's failing to upgrade to ODF 4.10: NAME DISPLAY VERSION REPLACES PHASE mcg-operator.v4.9.3 NooBaa Operator 4.9.3 mcg-operator.v4.9.2 Succeeded ocs-operator.v4.9.3 OpenShift Container Storage 4.9.3 ocs-operator.v4.9.2 Succeeded odf-operator.v4.10.0 OpenShift Data Foundation 4.10.0 odf-operator.v4.9.3 Pending odf-operator.v4.9.3 OpenShift Data Foundation 4.9.3 odf-operator.v4.9.2 Replacing https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/889/artifact/logs/failed_testcase_ocs_logs_1645754820/test_upgrade_ocs_logs/ocs_must_gather/quay-io-rhceph-dev-ocs-must-gather-sha256-4f471cfdf2ff928f13480f4356c953086306f2ff6c9bb677c361b72459ef0534/namespaces/openshift-storage/oc_output/csv/*view*/ message: 'installing: missing deployment with name=odf-operator-controller-manager' From what I see here: https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/889/artifact/logs/failed_testcase_ocs_logs_1645754820/test_upgrade_ocs_logs/ocp_must_gather/quay-io-openshift-origin-must-gather-sha256-17153e4bb8f0204cd15225fa33c2b15d11411e043f023c7d863136c430a20875/namespaces/openshift-storage/operators.coreos.com/clusterserviceversions/odf-operator.v4.10.0.yaml/*view*/ Version of all relevant components (if applicable): quay.io/rhceph-dev/ocs-registry:4.10.0-168 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Is there any workaround available to the best of your knowledge? Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? Can this issue reproducible? Can this issue reproduce from the UI? If this is a regression, please provide more details to justify this: Steps to Reproduce: 1. Install ODF 4.9 2. Upgrade to latest 4.10 build 3. Actual results: Expected results: Additional info: Logs: https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/889/artifact/logs/failed_testcase_ocs_logs_1645754820/test_upgrade_ocs_logs/
``` message: 'install strategy failed: Deployment.apps "odf-operator-controller-manager" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"odf-operator"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable' ``` Root cause: Olm is not able to upgrade the operator because of the conflict in the labels which were replaced recently because of another UI bug BZ2043034. I will update the fix after testing it out.
After discussing it with Afreen we decided to make the changes in the CSI addons and reverted the odf changes.
This BZ was caused by a fix for https://bugzilla.redhat.com/show_bug.cgi?id=2043034 We are reverting that fix, once the revert PR is merged this BZ can be moved to ON_QA
I see it's failing now here: https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/895 with: - message: 'constraints not satisfiable: clusterserviceversion odf-operator.v4.9.3 exists and is not referenced by a subscription, subscription odf-operator requires @existing/openshift-storage//odf-operator.v4.10.0, subscription odf-operator exists, @existing/openshift-storage//odf-operator.v4.10.0 and @existing/openshift-storage//odf-operator.v4.9.3 originate from package odf-operator' reason: ConstraintsNotSatisfiable https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/895/artifact/logs/failed_testcase_ocs_logs_1646113868/test_upgrade_ocs_logs/ocp_must_gather/quay-io-openshift-origin-must-gather-sha256-482074a79249dbe3d59fe9ec36f31499d27f4523d8909ab2e8e87e60a65b817a/namespaces/openshift-storage/operators.coreos.com/subscriptions/odf-operator.yaml/*view*/ So if build: 4.10.0-172 supposed to already fix that we still have some issue which might be different bug or it was not reverted/fixed properly.
Build #172 doesn't have the fix, that is why I didn't move the BZ to ON_QA Because of a build issue this build din't pick the latest commits, we need to wait for the next build.
What I see here it looks that issue was fixed: https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/909/console Marking as verified.