Bug 1907792
| Summary: | The `overrides` of the OperatorCondition cannot block the operator upgrade | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Jian Zhang <jiazha> |
| Component: | OLM | Assignee: | Evan Cordell <ecordell> |
| OLM sub component: | OLM | QA Contact: | Jian Zhang <jiazha> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | vdinh |
| Version: | 4.7 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-02-24 15:43:55 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
*** Bug 1909640 has been marked as a duplicate of this bug. *** The issue here is we use "OperatorUpgradeable" as the status. That is a mistake. We are correcting this to be "Upgradeable" as it is specified in the enhancement. Cluster version is 4.7.0-0.nightly-2021-01-04-215816
[root@preserve-olm-env data]# oc -n openshift-operator-lifecycle-manager exec catalog-operator-5b4b6587c4-2qgfb -- olm --version
OLM version: 0.17.0
git commit: 69dc076d610b033d3964ed40c4ac27ece3424851
1, Install the etcdoperator v0.9.2 with Manual approval
[root@preserve-olm-env data]# cat sub-etcd-cluster.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: etcd
namespace: openshift-operators
spec:
channel: clusterwide-alpha
installPlanApproval: Manual
name: etcd
source: community-operators
sourceNamespace: openshift-marketplace
startingCSV: etcdoperator.v0.9.2-clusterwide
[root@preserve-olm-env data]# oc create -f sub-etcd-cluster.yaml
subscription.operators.coreos.com/etcd created
2, Approve the v0.9.2 installation.
[root@preserve-olm-env data]# oc get ip
NAME CSV APPROVAL APPROVED
install-2b8z7 etcdoperator.v0.9.4-clusterwide Manual false
install-tcjbg etcdoperator.v0.9.2-clusterwide Manual true
[root@preserve-olm-env data]# oc get csv
NAME DISPLAY VERSION REPLACES PHASE
etcdoperator.v0.9.2-clusterwide etcd 0.9.2-clusterwide etcdoperator.v0.9.0 Installing
[root@preserve-olm-env data]# oc get operatorcondition
NAME AGE
etcdoperator.v0.9.2-clusterwide 38s
3, Add the `overrides` field to this OperatorCondition etcdoperator.v0.9.2-clusterwide.
...
spec:
deployments:
- etcd-operator
serviceAccounts:
- etcd-operator
overrides:
- message: Disbale the upgrade.
reason: upgradeIsNotSafe
status: "False"
type: Upgradeable
[root@preserve-olm-env data]# oc edit operatorcondition etcdoperator.v0.9.2-clusterwide
error: operatorconditions.operators.coreos.com "etcdoperator.v0.9.2-clusterwide" is invalid
operatorcondition.operators.coreos.com/etcdoperator.v0.9.2-clusterwide edited
4, Approve the v0.9.4 installation.
[root@preserve-olm-env data]# oc get ip
NAME CSV APPROVAL APPROVED
install-2b8z7 etcdoperator.v0.9.4-clusterwide Manual true
install-tcjbg etcdoperator.v0.9.2-clusterwide Manual true
[root@preserve-olm-env data]# oc get csv
NAME DISPLAY VERSION REPLACES PHASE
etcdoperator.v0.9.2-clusterwide etcd 0.9.2-clusterwide etcdoperator.v0.9.0 Replacing
etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide etcdoperator.v0.9.2-clusterwide Pending
As you can see, the v0.9.4 CSV in `Pending` status.
[root@preserve-olm-env data]# oc describe csv etcdoperator.v0.9.4-clusterwide
Name: etcdoperator.v0.9.4-clusterwide
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal RequirementsUnknown <invalid> operator-lifecycle-manager requirements not yet checked
Normal OperatorConditionNotUpgradeable <invalid> operator-lifecycle-manager operator is not upgradeable: The operator is not upgradeable: Disbale the upgrade.
5, Change the Upgradable status of the etcdoperator.v0.9.2-clusterwide to "True" from the "False".
[root@preserve-olm-env data]# oc get operatorcondition
NAME AGE
etcdoperator.v0.9.2-clusterwide 13m
etcdoperator.v0.9.4-clusterwide 10m
[root@preserve-olm-env data]# oc edit operatorcondition etcdoperator.v0.9.2-clusterwide
error: operatorconditions.operators.coreos.com "etcdoperator.v0.9.2-clusterwide" is invalid
operatorcondition.operators.coreos.com/etcdoperator.v0.9.2-clusterwide edited
[root@preserve-olm-env data]# oc get operatorcondition
NAME AGE
etcdoperator.v0.9.4-clusterwide 11m
[root@preserve-olm-env data]# oc get csv
NAME DISPLAY VERSION REPLACES PHASE
etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide etcdoperator.v0.9.2-clusterwide Succeeded
[root@preserve-olm-env data]#
[root@preserve-olm-env data]# oc get pods
NAME READY STATUS RESTARTS AGE
etcd-operator-5b8f9c4fbc-jhn5d 3/3 Running 0 30s
etcd-operator-869664477d-t57vn 0/3 Terminating 0 15m
The v0.9.2 can be upgraded to v0.9.4 successfully, looks good to me, verify it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633 |
Description of problem: The `overrides` of the OperatorCondition cannot block the operator upgrade. Version-Release number of selected component (if applicable): [root@preserve-olm-env data]# oc -n openshift-operator-lifecycle-manager exec catalog-operator-bb89cfd76-jbdqf -- olm --version OLM version: 0.17.0 git commit: 4b66803055a8ab611447c33ed86e755ad39cb313 How reproducible: always Steps to Reproduce: 1. Create a CatalogSource to consume the image: quay.io/olmqe/etcd-index:0.9.2-oc [root@preserve-olm-env data]# cat cs-poll.yaml apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: oc-test spec: displayName: Jian Test sourceType: grpc image: quay.io/olmqe/etcd-index:0.9.2-oc updateStrategy: registryPoll: interval: 10m 2. Subscribe the etcdoperator.v0.9.2 with automatic upgrade. [root@preserve-olm-env data]# cat sub-new.yaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: etcd namespace: default spec: channel: "4.6" installPlanApproval: Automatic name: etcd source: oc-test sourceNamespace: openshift-marketplace startingCSV: etcdoperator.v0.9.2 [root@preserve-olm-env data]# oc create -f sub-new.yaml subscription.operators.coreos.com/etcd created [root@preserve-olm-env data]# oc get sub -A NAMESPACE NAME PACKAGE SOURCE CHANNEL default etcd etcd oc-test 4.6 [root@preserve-olm-env data]# oc get ip NAME CSV APPROVAL APPROVED install-bgrtm etcdoperator.v0.9.2 Automatic true [root@preserve-olm-env data]# oc get csv NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.2 etcd 0.9.2 Succeeded [root@preserve-olm-env data]# oc get operatorcondition NAME AGE etcdoperator.v0.9.2 9m 3. Modify the OperatorCondition etcdoperator.v0.9.2, add the `overrrides` field: Set the `Upgradeable` to `False`. [root@preserve-olm-env data]# oc get operatorcondition etcdoperator.v0.9.2 -o yaml apiVersion: operators.coreos.com/v1 kind: OperatorCondition ... spec: deployments: - etcd-operator overrides: - lastTransitionTime: "2020-12-15T23:15:55Z" message: Disbale the upgrade. reason: upgradeIsNotSafe status: "False" type: Upgradeable serviceAccounts: - etcd-operator 4, Add the etcd 0.9.4 bundle to this index image: quay.io/olmqe/etcd-index:0.9.2-oc [root@preserve-olm-env etcd]# opm index add -u docker -b quay.io/olmqe/etcd-bundle:0.9.4 -f quay.io/olmqe/etcd-index:0.9.2-oc -t quay.io/olmqe/etcd-index:0.9.2-oc --mode semver INFO[0000] building the index bundles="[quay.io/olmqe/etcd-bundle:0.9.4]" ... INFO[0002] [docker build -f index.Dockerfile923284778 -t quay.io/olmqe/etcd-index:0.9.2-oc .] bundles="[quay.io/olmqe/etcd-bundle:0.9.4]" [root@preserve-olm-env etcd]# docker push quay.io/olmqe/etcd-index:0.9.2-oc ... 5, Wait for the CatalogSource upgrade. [root@preserve-olm-env data]# oc get pods -w NAME READY STATUS RESTARTS AGE 8cc7f4add666d4b2bd624f5b2ebc5f0dc940bb77327e333b208b8453f377p5p 0/1 Completed 0 9m58s ... oc-test-6bp9r 0/1 Pending 0 0s community-operators-rzh9z 0/1 Pending 0 0s oc-test-6bp9r 0/1 Pending 0 0s community-operators-rzh9z 0/1 Pending 0 0s oc-test-6bp9r 0/1 ContainerCreating 0 0s community-operators-rzh9z 0/1 ContainerCreating 0 0s community-operators-rzh9z 0/1 ContainerCreating 0 2s oc-test-6bp9r 0/1 ContainerCreating 0 2s 6, Check if the installed 0.9.2 etcd operator upgrade. Actual results: The etcdoperatorv0.9.2 was upgraded to etcdoperator.v0.9.4, but it should not. [root@preserve-olm-env data]# oc get csv -n default NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.4 etcd 0.9.4 etcdoperator.v0.9.2 Succeeded [root@preserve-olm-env data]# oc get operatorcondition -n default NAME AGE etcdoperator.v0.9.4 110s [root@preserve-olm-env data]# oc get operatorcondition -n default etcdoperator.v0.9.4 -o yaml apiVersion: operators.coreos.com/v1 kind: OperatorCondition ... spec: deployments: - etcd-operator serviceAccounts: - etcd-operator Expected results: The etcdoperatorv0.9.2 cannot be upgraded to 0.9.4 since the `Upgradeable` of OperatorCondition is false. Additional info: