Bug 1910160 - If two OperatorConditions include the same deployments they will keep updating the deployment's containers with the OPERATOR_CONDITION_NAME Environment Variable
Summary: If two OperatorConditions include the same deployments they will keep updatin...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: OLM
Version: 4.7
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.7.0
Assignee: Alexander Greene
QA Contact: Jian Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-22 22:01 UTC by Alexander Greene
Modified: 2021-02-24 15:48 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Most operators managed by OLM do not change the deployment name of their operators between versions. Consequence: OLM injects an environment variable into Deployments defined in a CSV with the name of the OperatorCondition that they own. When upgrading an operator, two OperatorConditions can exist that define the same deployment. Fix: The OperatorCondition reconciler will only update adeployments owned by a CSV with the same name of the OperatorCondition. Result: Two OperatorConditions will not continuously update a deployment that the both define.
Clone Of:
Environment:
Last Closed: 2021-02-24 15:48:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github operator-framework operator-lifecycle-manager pull 1900 0 None closed Bug 1910160: Multiple operatorconditions should not update the same deployment 2021-02-11 17:41:52 UTC
Github operator-framework operator-lifecycle-manager pull 1942 0 None closed Bug 1910160: Update OperatorCondition controller to watch deps 2021-02-11 17:41:52 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:48:56 UTC

Description Alexander Greene 2020-12-22 22:01:11 UTC
Description of problem:
If two OperatorConditions include the same deployments they will keep updating the deployment's containers with the OPERATOR_CONDITION_NAME Environment Variable.

Version-Release number of selected component (if applicable):
4.7

How reproducible:
Frequently while upgrading CSVs.

Steps to Reproduce:
1. Install csv A which defines a deployment.
2. Install csvB which replaces csvB which defines a deployment with the same name as the one defined in step 1.

Actual results:
While csvB is being installed the OPERATOR_CONDITION_NAME environment variable switches between values csvA.Name and csvB.Name.

Expected results:
The OPERATOR_CONDITION_NAME should always be csvA.Name until csvB is being installed, at which point the OPERATOR_CONDITION_NAME should swithc the csvB.Name

Comment 4 Jian Zhang 2021-01-06 02:25:57 UTC
Cluster version is 4.7.0-0.nightly-2021-01-04-215816
[root@preserve-olm-env data]# oc  -n openshift-operator-lifecycle-manager exec catalog-operator-5b4b6587c4-2qgfb -- olm --version
OLM version: 0.17.0
git commit: 69dc076d610b033d3964ed40c4ac27ece3424851


1, Install the etcdoperator v0.9.2 with Manual approval
[root@preserve-olm-env data]# cat sub-etcd-cluster.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: etcd
  namespace: openshift-operators
spec:
  channel: clusterwide-alpha
  installPlanApproval: Manual
  name: etcd
  source: community-operators
  sourceNamespace: openshift-marketplace
  startingCSV: etcdoperator.v0.9.2-clusterwide
[root@preserve-olm-env data]# oc create -f sub-etcd-cluster.yaml 
subscription.operators.coreos.com/etcd created

2, Approve the v0.9.2 installation.
[root@preserve-olm-env data]# oc get ip
NAME            CSV                               APPROVAL   APPROVED
install-2b8z7   etcdoperator.v0.9.4-clusterwide   Manual     false
install-tcjbg   etcdoperator.v0.9.2-clusterwide   Manual     true
[root@preserve-olm-env data]# oc get csv
NAME                              DISPLAY   VERSION             REPLACES              PHASE
etcdoperator.v0.9.2-clusterwide   etcd      0.9.2-clusterwide   etcdoperator.v0.9.0   Installing
[root@preserve-olm-env data]# oc get operatorcondition
NAME                              AGE
etcdoperator.v0.9.2-clusterwide   38s


3, Check the v0.9.2 deployment.
[root@preserve-olm-env data]# oc get csv
 NAME                              DISPLAY   VERSION             REPLACES              PHASE
etcdoperator.v0.9.2-clusterwide   etcd      0.9.2-clusterwide   etcdoperator.v0.9.0   Succeeded
[root@preserve-olm-env data]#  
[root@preserve-olm-env data]# 
[root@preserve-olm-env data]# oc get deployment
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
etcd-operator   1/1     1            1           3m17s

[root@preserve-olm-env data]# oc get deployment etcd-operator -o yaml |grep OPERATOR_CONDITION_NAME

But, there is no "OPERATOR_CONDITION_NAME" env var in it. Change the status to "ASSIGNED".

Comment 9 Jian Zhang 2021-01-15 06:09:47 UTC
Cluster version is 4.7.0-0.nightly-2021-01-14-211319

[root@preserve-olm-env data]# oc -n openshift-operator-lifecycle-manager exec catalog-operator-5b48b7c6b9-rbcxr  -- olm --version
OLM version: 0.17.0
git commit: 4a9d02bec674e0398eb5540bdffa13ffd96cd7eb

1, Install the etcdoperator v0.9.2 with Manual approval
[root@preserve-olm-env data]# cat sub-etcd-cluster.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: etcd
  namespace: openshift-operators
spec:
  channel: clusterwide-alpha
  installPlanApproval: Manual
  name: etcd
  source: community-operators
  sourceNamespace: openshift-marketplace
  startingCSV: etcdoperator.v0.9.2-clusterwide
[root@preserve-olm-env data]# 
[root@preserve-olm-env data]# oc create -f sub-etcd-cluster.yaml 
subscription.operators.coreos.com/etcd created

2, Approve the v0.9.2 installation.

[root@preserve-olm-env data]# oc get ip
NAME            CSV                               APPROVAL   APPROVED
install-9rxr7   etcdoperator.v0.9.2-clusterwide   Manual     true
install-sxnzd   etcdoperator.v0.9.4-clusterwide   Manual     false

3, Check the v0.9.2 deployment. The OPERATOR_CONDITION_NAME value is "etcdoperator.v0.9.2-clusterwide", looks good.
[root@preserve-olm-env data]# oc get pod
NAME                             READY   STATUS    RESTARTS   AGE
etcd-operator-66c87f98c8-q2fvr   3/3     Running   0          4m3s
[root@preserve-olm-env data]# oc get deployment
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
etcd-operator   1/1     1            1           4m20s
[root@preserve-olm-env data]# oc get operatorcondition  etcdoperator.v0.9.2-clusterwide -o yaml
apiVersion: operators.coreos.com/v1
kind: OperatorCondition
...
spec:
  deployments:
  - etcd-operator
  serviceAccounts:
  - etcd-operator


[root@preserve-olm-env data]# oc get deployment -o yaml|grep OPERATOR_CONDITION_NAME -A1
--
          - name: OPERATOR_CONDITION_NAME
            value: etcdoperator.v0.9.2-clusterwide
--
          - name: OPERATOR_CONDITION_NAME
            value: etcdoperator.v0.9.2-clusterwide
--
          - name: OPERATOR_CONDITION_NAME
            value: etcdoperator.v0.9.2-clusterwide

4, Approve the v0.9.4 installation. The OPERATOR_CONDITION_NAME value updated to "etcdoperator.v0.9.4-clusterwide"
[root@preserve-olm-env data]# oc get ip
NAME            CSV                               APPROVAL   APPROVED
install-9rxr7   etcdoperator.v0.9.2-clusterwide   Manual     true
install-sxnzd   etcdoperator.v0.9.4-clusterwide   Manual     true
[root@preserve-olm-env data]# oc get csv
NAME                              DISPLAY   VERSION             REPLACES                          PHASE
etcdoperator.v0.9.2-clusterwide   etcd      0.9.2-clusterwide   etcdoperator.v0.9.0               Replacing
etcdoperator.v0.9.4-clusterwide   etcd      0.9.4-clusterwide   etcdoperator.v0.9.2-clusterwide   Installing
[root@preserve-olm-env data]# oc get operatorcondition
NAME                              AGE
etcdoperator.v0.9.4-clusterwide   17s
[root@preserve-olm-env data]# oc get deployment
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
etcd-operator   1/1     1            1           7m46s
[root@preserve-olm-env data]# oc get deployment -o yaml|grep OPERATOR_CONDITION_NAME -A1
--
          - name: OPERATOR_CONDITION_NAME
            value: etcdoperator.v0.9.4-clusterwide
--
          - name: OPERATOR_CONDITION_NAME
            value: etcdoperator.v0.9.4-clusterwide
--
          - name: OPERATOR_CONDITION_NAME
            value: etcdoperator.v0.9.4-clusterwide

Looks good to me, verify it.

Comment 12 errata-xmlrpc 2021-02-24 15:48:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.