Bug 1825088
| Summary: | The installed operator cannot be upgraded after the cluster upgraded to 4.4 from 4.3 | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Anping Li <anli> |
| Component: | OLM | Assignee: | Alexander Greene <agreene> |
| OLM sub component: | OLM | QA Contact: | Jian Zhang <jiazha> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | high | ||
| Priority: | urgent | CC: | agreene, aos-bugs, bbennett, vdinh |
| Version: | 4.4 | Keywords: | Regression |
| Target Milestone: | --- | ||
| Target Release: | 4.5.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: The Generation field was not marked as optional.
Consequence: Custom Resources created without the generation field failed schema validation.
Fix: Mark the generation field as an optional field.
Result: InstallPlans may be created without the generation field.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-13 17:27:59 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1825349, 1825845 | ||
Remove testblocker for the csv be updated on the other cluster. The generation field is no longer marked as required as this PR was merged: https://github.com/operator-framework/operator-lifecycle-manager/pull/1464/ 1, Install 4.5 cluster that contains the fixed PR, as follows:
Cluster version is 4.5.0-0.nightly-2020-04-21-103613
mac:~ jianzhang$ oc exec catalog-operator-79c4444645-mqgfh -- olm --version
OLM version: 0.14.2
git commit: 9f126de212818b9f6477cfefec64df1898f8419d
2, Subscribe an operator on the Web console. For example, the cluster-logging operator, 4.3 version.
mac:~ jianzhang$ oc get sub -n default
NAME PACKAGE SOURCE CHANNEL
cluster-logging cluster-logging redhat-operators 4.3
3, Check its InstallPlan object. As we can see, the `generation` field exists.
mac:~ jianzhang$ oc get ip -n default
NAME CSV APPROVAL APPROVED
install-9j874 clusterlogging.4.3.13-202004131016 Automatic true
mac:~ jianzhang$ oc get ip install-9j874 -n default -o=jsonpath='{.spec}'
map[approval:Automatic approved:true clusterServiceVersionNames:[clusterlogging.4.3.13-202004131016] generation:1]
4, Modify this InstallPlan object, remove the `generation` field.
mac:~ jianzhang$ oc edit ip install-9j874
installplan.operators.coreos.com/install-9j874 edited
mac:~ jianzhang$ oc get ip install-9j874 -n default -o=jsonpath='{.spec}'
map[approval:Automatic approved:true clusterServiceVersionNames:[clusterlogging.4.3.13-202004131016]]
5, Modify the subscribe object, change the 4.3 channel to 4.4.
mac:~ jianzhang$ oc edit sub cluster-logging
subscription.operators.coreos.com/cluster-logging edited
6, Check the InstallPlan/CSV objects.
mac:~ jianzhang$ oc get ip
NAME CSV APPROVAL APPROVED
install-9j874 clusterlogging.4.3.13-202004131016 Automatic true
install-xk8lz clusterlogging.4.4.0-202004091036 Automatic true
mac:~ jianzhang$ oc get ip install-xk8lz -o=jsonpath='{.spec}'
map[approval:Automatic approved:true clusterServiceVersionNames:[clusterlogging.4.4.0-202004091036] generation:2]
mac:~ jianzhang$
mac:~ jianzhang$ oc get ip install-9j874 -o=jsonpath='{.spec}'
map[approval:Automatic approved:true clusterServiceVersionNames:[clusterlogging.4.3.13-202004131016]]
mac:~ jianzhang$ oc get csv
NAME DISPLAY VERSION REPLACES PHASE
clusterlogging.4.3.13-202004131016 Cluster Logging 4.3.13-202004131016 Pending
clusterlogging.4.4.0-202004091036 Cluster Logging 4.4.0-202004091036 clusterlogging.4.3.13-202004131016 Pending
mac:~ jianzhang$ oc logs catalog-operator-79c4444645-mqgfh |grep install-9j874 |grep update |grep generation
mac:~ jianzhang$
As we can see, the new InstallPlan "install-xk8lz "and CSV "clusterlogging.4.4.0-202004091036" generated even if there is no `generation` field in the old InstallPlan "install-9j874". LGTM, verify it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409 |
Description of problem: The installed operator cannot be upgraded due to the lack of 'generation' field of the InstallPlan after the cluster upgraded to 4.4 from 4.3, errors as follows: E0416 16:32:18.613181 1 queueinformer_operator.go:290] sync {"update" "openshift-logging/install-tmllk"} failed: error updating InstallPlan status: InstallPlan.operators.coreos.com "install-tmllk" is invalid: spec.generation: Required value mac:~ jianzhang$ oc get ip -n openshift-logging install-tmllk -o=jsonpath='{.spec}' map[approval:Automatic approved:true clusterServiceVersionNames:[clusterlogging.4.3.10-202004010435]] Version-Release number of selected component (if applicable): 4.4.0-0.nightly-2020-04-16-094508 OLM version: $oc exec catalog-operator-56b67964ff-cchtq -- olm --version OLM version: 0.14.2 git commit: 2248cc6bea92a03b450e773b84ef8fa924c9c1c9 How reproducible: This cluster. Steps to Reproduce: 1, Install OCP 4.3 2. Deploy cluster-logging from redhat-operators 3, Upgrade the OCP 4.3 to 4.4 4. Create a OperatorSource to use QE app registry. mac:~ jianzhang$ oc get operatorsource -n openshift-marketplace qe-app-registry -o yaml apiVersion: operators.coreos.com/v1 kind: OperatorSource metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"operators.coreos.com/v1","kind":"OperatorSource","metadata":{"annotations":{},"name":"qe-app-registry","namespace":"openshift-marketplace"},"spec":{"authorizationToken":{"secretName":"qesecret"},"endpoint":"https://quay.io/cnr","registryNamespace":"openshift-qe-optional-operators","type":"appregistry"}} creationTimestamp: "2020-04-16T06:50:36Z" finalizers: - finalizer.operatorsources.operators.coreos.com generation: 75 name: qe-app-registry namespace: openshift-marketplace resourceVersion: "675372" selfLink: /apis/operators.coreos.com/v1/namespaces/openshift-marketplace/operatorsources/qe-app-registry uid: 9389b6aa-7fae-11ea-8773-02505d469966 spec: authorizationToken: secretName: qesecret endpoint: https://quay.io/cnr registryNamespace: openshift-qe-optional-operators type: appregistry status: currentPhase: lastTransitionTime: "2020-04-17T01:01:33Z" lastUpdateTime: "2020-04-17T01:01:33Z" phase: message: The object has been successfully reconciled name: Succeeded packages: openshifttemplateservicebroker,metering-ocp,nfd,ptp-operator,elasticsearch-operator,cluster-logging,local-storage-operator,sriov-network-operator,clusterresourceoverride,cluster-kube-descheduler-operator,openshiftansibleservicebroker 3. Check the csv if upgraded successfully. Actual results: # the csv wasn't upgraded to 4.4.0 oc get csv NAME DISPLAY VERSION REPLACES PHASE clusterlogging.4.3.10-202004010435 Cluster Logging 4.3.10-202004010435 Succeeded elasticsearch-operator.4.3.10-202003311428 Elasticsearch Operator 4.3.10-202003311428 Succeeded Expected result # the csv is upgraded to 4.4.0-202004150031 oc get csv NAME DISPLAY VERSION REPLACES PHASE clusterlogging.4.4.0-202004150031 Cluster Logging 4.4.0-202004150031 Succeeded elasticsearch-operator.4.4.0-202004150031 Elasticsearch Operator 4.4.0-202004150031 Succeeded Addtional info: