Bug 1921953
| Summary: | ClusterServiceVersion property inference does not infer package and version | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Ben Luddy <bluddy> | |
| Component: | OLM | Assignee: | Ben Luddy <bluddy> | |
| OLM sub component: | OLM | QA Contact: | kuiwang | |
| Status: | CLOSED ERRATA | Docs Contact: | ||
| Severity: | high | |||
| Priority: | high | CC: | bluddy, Jiaming.Hu, krizza, vdinh | |
| Version: | 4.6 | Keywords: | Triaged | |
| Target Milestone: | --- | |||
| Target Release: | 4.8.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: |
Cause: Operators that were installed on pre-4.6 clusters are not identified as coming from a given operator package for the purposes of dependency resolution and upgrade selection.
Consequence: Existing operator installations can conflict with the criteria of their own Subscription, blocking upgrades and dependency resolution within the namespace.
Fix: Infer package name and version for operators that are referenced by a Subscription.
Result: Upgrades and dependency resolution proceed as expected in the presence of operators installed pre-4.6.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1929904 (view as bug list) | Environment: | ||
| Last Closed: | 2021-07-27 22:37:34 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1929904 | |||
Marking for upcoming sprint and removing from the 4.7.0 release. This is not a blocker for 4.7 and will be addressed in a future sprint. verify it on 4.8. Fail
--
[root@preserve-olm-env 1921953]# oc get pod -n openshift-operator-lifecycle-manager
NAME READY STATUS RESTARTS AGE
catalog-operator-679db8fbc4-qnqv2 1/1 Running 0 44m
olm-operator-79cb7777cc-klkdx 1/1 Running 0 44m
packageserver-f467b6887-58cfs 1/1 Running 0 37m
packageserver-f467b6887-6m7qn 1/1 Running 0 37m
[root@preserve-olm-env 1921953]# oc exec catalog-operator-679db8fbc4-qnqv2 -n openshift-operator-lifecycle-manager -- olm --version
OLM version: 0.17.0
git commit: 034b4001f752c23aa5a35744f7a8536afab0cd63
[root@preserve-olm-env 1921953]# oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.8.0-0.nightly-2021-02-19-182344 True False 19m Cluster version is 4.8.0-0.nightly-2021-02-19-182344
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# cat res.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: ibm-common-service-catalog
namespace: default
spec:
image: docker.io/ibmcom/ibm-common-service-catalog:latest
sourceType: grpc
---
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: ibm-operator-catalog
namespace: default
spec:
image: docker.io/ibmcom/ibm-operator-catalog:latest
sourceType: grpc
---
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: test-operator-group
namespace: default
spec: {}
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: test-ibm-common-service-operator
namespace: default
spec:
channel: stable-v1
installPlanApproval: Automatic
name: ibm-common-service-operator
source: ibm-common-service-catalog
sourceNamespace: default
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl create -f res.yaml
catalogsource.operators.coreos.com/ibm-common-service-catalog created
catalogsource.operators.coreos.com/ibm-operator-catalog created
operatorgroup.operators.coreos.com/test-operator-group created
subscription.operators.coreos.com/test-ibm-common-service-operator created
[root@preserve-olm-env 1921953]# kubectl get csv
NAME DISPLAY VERSION REPLACES PHASE
ibm-common-service-operator.v3.6.3 IBM Cloud Platform Common Services 3.6.3 ibm-common-service-operator.v3.6.2 Succeeded
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl get csv ibm-common-service-operator.v3.6.3 -o yaml|grep "operatorframework.io/properties"
operatorframework.io/properties: '{"properties":[{"type":"olm.gvk","value":{"group":"operator.ibm.com","kind":"CommonService","version":"v3"}},{"type":"olm.package","value":{"packageName":"ibm-common-service-operator","version":"3.6.3"}}]}'
f:operatorframework.io/properties: {}
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl patch -n default csv ibm-common-service-operator.v3.6.3 --type=json -p='[{"op":"remove","path":"/metadata/annotations/operatorframework.io~1properties"}]'
clusterserviceversion.operators.coreos.com/ibm-common-service-operator.v3.6.3 patched
[root@preserve-olm-env 1921953]# kubectl get csv ibm-common-service-operator.v3.6.3 -o yaml|grep "operatorframework.io/properties"
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# cat sub.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: test-ibm-integration-platform-navigator
namespace: default
spec:
channel: v4.1-eus
installPlanApproval: Automatic
name: ibm-integration-platform-navigator
source: ibm-operator-catalog
sourceNamespace: default
[root@preserve-olm-env 1921953]# kubectl create -f sub.yaml
subscription.operators.coreos.com/test-ibm-integration-platform-navigator created
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl get csv
NAME DISPLAY VERSION REPLACES PHASE
ibm-common-service-operator.v3.6.3 IBM Cloud Platform Common Services 3.6.3 ibm-common-service-operator.v3.6.2 Succeeded
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl get -n default event --sort-by=.metadata.creationTimestamp | grep ResolutionFailed | tail -1
5m31s Warning ResolutionFailed namespace/default (combined from similar events): constraints not satisfiable: ibm-operator-catalog/default/v4.1-eus/ibm-integration-platform-navigator.v4.1.0 requires at least one of ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.3, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.0, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.0, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.0, subscription test-ibm-common-service-operator requires @existing/default//ibm-common-service-operator.v3.6.3, subscription test-ibm-common-service-operator exists, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.4.0, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.5.1, @existing/default//ibm-common-service-operator.v3.6.3, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.4.3, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.5.2, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.0, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.4.1, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.5.0, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.3, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.4.2, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.5.6, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.5.5, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.3.0, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.5.4 and ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.5.3 provide CommonService (operator.ibm.com/v3), subscription test-ibm-integration-platform-navigator requires ibm-operator-catalog/default/v4.1-eus/ibm-integration-platform-navigator.v4.1.0, subscription test-ibm-integration-platform-navigator exists
[root@preserve-olm-env 1921953]#
--
This is linked to the wrong PR. No fix for this BZ has merged yet. verify it on 4.8. LGTM
--
[root@preserve-olm-env 1921953]# oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.8.0-0.nightly-2021-02-23-160229 True False 5m5s Cluster version is 4.8.0-0.nightly-2021-02-23-160229
[root@preserve-olm-env 1921953]# oc get pod -n openshift-operator-lifecycle-manager
NAME READY STATUS RESTARTS AGE
catalog-operator-5bcc4c469b-pmlfr 1/1 Running 0 35m
olm-operator-9f454ff4d-xxp7q 1/1 Running 0 35m
packageserver-5558bbd88f-45xs8 1/1 Running 0 27m
packageserver-5558bbd88f-j2clv 1/1 Running 0 27m
[root@preserve-olm-env 1921953]# oc exec catalog-operator-5bcc4c469b-pmlfr -n openshift-operator-lifecycle-manager -- olm --version
OLM version: 0.17.0
git commit: fd0ce87079d83588ad4c6e74f3175edf340e28a8
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# cat res.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: ibm-common-service-catalog
namespace: default
spec:
image: docker.io/ibmcom/ibm-common-service-catalog:latest
sourceType: grpc
---
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: ibm-operator-catalog
namespace: default
spec:
image: docker.io/ibmcom/ibm-operator-catalog:latest
sourceType: grpc
---
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: test-operator-group
namespace: default
spec: {}
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: test-ibm-common-service-operator
namespace: default
spec:
channel: stable-v1
installPlanApproval: Automatic
name: ibm-common-service-operator
source: ibm-common-service-catalog
sourceNamespace: default
[root@preserve-olm-env 1921953]# kubectl create -f res.yaml
catalogsource.operators.coreos.com/ibm-common-service-catalog created
catalogsource.operators.coreos.com/ibm-operator-catalog created
operatorgroup.operators.coreos.com/test-operator-group created
subscription.operators.coreos.com/test-ibm-common-service-operator created
[root@preserve-olm-env 1921953]# kubectl get csv
NAME DISPLAY VERSION REPLACES PHASE
ibm-common-service-operator.v3.6.3 IBM Cloud Platform Common Services 3.6.3 ibm-common-service-operator.v3.6.2 Succeeded
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl get csv ibm-common-service-operator.v3.6.3 -o yaml|grep "operatorframework.io/properties"
operatorframework.io/properties: '{"properties":[{"type":"olm.gvk","value":{"group":"operator.ibm.com","kind":"CommonService","version":"v3"}},{"type":"olm.package","value":{"packageName":"ibm-common-service-operator","version":"3.6.3"}}]}'
f:operatorframework.io/properties: {}
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# kubectl patch -n default csv ibm-common-service-operator.v3.6.3 --type=json -p='[{"op":"remove","path":"/metadata/annotations/operatorframework.io~1properties"}]'
clusterserviceversion.operators.coreos.com/ibm-common-service-operator.v3.6.3 patched
[root@preserve-olm-env 1921953]# kubectl get csv ibm-common-service-operator.v3.6.3 -o yaml|grep "operatorframework.io/properties"
[root@preserve-olm-env 1921953]#
[root@preserve-olm-env 1921953]# cat sub.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: test-ibm-integration-platform-navigator
namespace: default
spec:
channel: v4.1-eus
installPlanApproval: Automatic
name: ibm-integration-platform-navigator
source: ibm-operator-catalog
sourceNamespace: default
[root@preserve-olm-env 1921953]# kubectl create -f sub.yaml
subscription.operators.coreos.com/test-ibm-integration-platform-navigator created
[root@preserve-olm-env 1921953]# kubectl get csv
NAME DISPLAY VERSION REPLACES PHASE
ibm-common-service-operator.v3.6.3 IBM Cloud Platform Common Services 3.6.3 ibm-common-service-operator.v3.6.2 Succeeded
ibm-integration-platform-navigator.v4.1.0 IBM Cloud Pak for Integration Platform Navigator 4.1.0 Succeeded
[root@preserve-olm-env 1921953]#
--
Will this fix be picked to OCP 4.6? Yes, it's working its way back to 4.6 now. It's currently waiting to be accepted into 4.7 (see https://bugzilla.redhat.com/show_bug.cgi?id=1929904). Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |
All ClusterServiceVersions installed on OCP 4.6+ are automatically annotated with a snapshot of relevant information (properties) from the catalog they were installed from. During dependency resolution, this annotation determines whether or not a CSV is able to satisfy the dependencies of other operators. If a ClusterServiceVersion is missing this annotation (which is the case if it was installed prior to an upgrade to OCP 4.6+), some of this information is inferred by inspecting the ClusterServiceVersion's spec. Because "package" is not part of the CSV spec, it is not inferred. If a user later tries to install an operator that declares a direct package dependency (introduced in 4.6+), an unannotated (pre-4.6) CSV can never be chosen to satisfy the dependency. It is sometimes possible to infer package by finding a Subscription that relates an unannotated CSV to a CatalogSource. It's not possible in cases where there is no associated Subscription or if the operator in question is no longer present in the catalog. Steps to Reproduce: $ cat << EOF | kubectl create -f- apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: ibm-common-service-catalog namespace: default spec: image: docker.io/ibmcom/ibm-common-service-catalog:latest sourceType: grpc --- apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: ibm-operator-catalog namespace: default spec: image: docker.io/ibmcom/ibm-operator-catalog:latest sourceType: grpc --- apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: test-operator-group namespace: default spec: {} --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: test-ibm-common-service-operator namespace: default spec: channel: stable-v1 installPlanApproval: Automatic name: ibm-common-service-operator source: ibm-common-service-catalog sourceNamespace: default EOF catalogsource.operators.coreos.com/ibm-common-service-catalog created catalogsource.operators.coreos.com/ibm-operator-catalog created operatorgroup.operators.coreos.com/test-operator-group created subscription.operators.coreos.com/test-ibm-common-service-operator created $ kubectl get csv -w # Wait until this CSV is installed! NAME DISPLAY VERSION REPLACES PHASE ibm-common-service-operator.v3.6.0 IBM Cloud Platform Common Services 3.6.0 ibm-common-service-operator.v3.5.6 Succeeded $ kubectl patch -n default csv ibm-common-service-operator.v3.6.3 --type=json -p='[{"op":"remove","path":"/metadata/annotations/operatorframework.io~1properties"}]' clusterserviceversion.operators.coreos.com/ibm-common-service-operator.v3.6.3 patched $ cat << EOF | kubectl create -f- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: test-ibm-integration-platform-navigator namespace: default spec: channel: v4.1-eus installPlanApproval: Automatic name: ibm-integration-platform-navigator source: ibm-operator-catalog sourceNamespace: default EOF subscription.operators.coreos.com/test-ibm-integration-platform-navigator created $ kubectl get csv -w # Observe that ibm-integration-platform-navigator is never installed. NAME DISPLAY VERSION REPLACES PHASE ibm-common-service-operator.v3.6.3 IBM Cloud Platform Common Services 3.6.3 ibm-common-service-operator.v3.6.0 Succeeded Expected Results: ibm-integration-platform-navigator is installed after a few moments. Actual Results: ibm-integration-platform-navigator is never installed. You will probably also see a big event explaining why it thinks the installation can't succeed, like this one: $ kubectl get -n default event --sort-by=.metadata.creationTimestamp | grep ResolutionFailed | tail -1 2m55s Warning ResolutionFailed namespace/default constraints not satisfiable: gvkunique/operator.ibm.com/v3/CommonService permits at most 1 of cs-catalog/olm/dev/ibm-common-service-operator.v3.5.0, cs-catalog/olm/beta/ibm-common-service-operator.v3.4.1, @existing/default//ibm-common-service-operator.v3.6.3, cs-catalog/olm/dev/ibm-common-service-operator.v3.5.3, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.4.3, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.6.3, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.5.5, cs-catalog/olm/dev/ibm-common-service-operator.v3.6.2, cs-catalog/olm/beta/ibm-common-service-operator.v3.4.0, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.5.4, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.0, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.4.1, cs-catalog/olm/dev/ibm-common-service-operator.v3.4.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.5.1, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.3, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.3.0, cs-catalog/olm/beta/ibm-common-service-operator.v3.4.3, cs-catalog/olm/beta/ibm-common-service-operator.v3.6.0, cs-catalog/olm/dev/ibm-common-service-operator.v3.5.6, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.5.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.4.0, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.2, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.6.1, cs-catalog/olm/dev/ibm-common-service-operator.v3.5.4, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.5.0, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.4.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.5.3, cs-catalog/olm/dev/ibm-common-service-operator.v3.5.1, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.5.5, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.3.0, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.5.6, cs-catalog/olm/beta/ibm-common-service-operator.v3.5.2, the subscription "ibm-common-service-operator" exists, ibm-common-service-operator requires at least one of @existing/default//ibm-common-service-operator.v3.6.3, ibm-operator-catalog/default/v4.1-eus/ibm-integration-platform-navigator.v4.1.0 requires at least one of ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.3, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.0, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.0, ibm-common-service-catalog/default/dev/ibm-common-service-operator.v3.6.2, ibm-common-service-catalog/default/stable-v1/ibm-common-service-operator.v3.6.1, ibm-common-service-catalog/default/beta/ibm-common-service-operator.v3.6.0, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.6.3, cs-catalog/olm/dev/ibm-common-service-operator.v3.6.2, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.6.1, cs-catalog/olm/beta/ibm-common-service-operator.v3.6.0, cs-catalog/olm/dev/ibm-common-service-operator.v3.6.2, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.6.1, cs-catalog/olm/beta/ibm-common-service-operator.v3.6.0, cs-catalog/olm/dev/ibm-common-service-operator.v3.6.2, cs-catalog/olm/stable-v1/ibm-common-service-operator.v3.6.1, cs-catalog/olm/beta/ibm-common-service-operator.v3.6.0, ibm-integration-platform-navigator requires at least one of ibm-operator-catalog/default/v4.1-eus/ibm-integration-platform-navigator.v4.1.0, the subscription "ibm-integration-platform-navigator" exists