Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1701468

Summary: Fail to deploy the same operator with different ways(clusterwide, singlenamespace) in the same cluster
Product: OpenShift Container Platform Reporter: Jian Zhang <jiazha>
Component: OLMAssignee: Evan Cordell <ecordell>
OLM sub component: OperatorHub QA Contact: Jian Zhang <jiazha>
Status: CLOSED WONTFIX Docs Contact:
Severity: medium    
Priority: high CC: bandrade, chezhang, dyan, jfan, scolange, zitang
Version: 4.1.0Keywords: Reopened
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-29 14:41:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jian Zhang 2019-04-19 07:02:08 UTC
Description of problem:
Got the "InterOperatorGroupOwnerConflict" when installing the clusterwide etcd operator after the singlenamespace etcd operator installed.

Version-Release number of selected component (if applicable):
Cluster version is 4.1.0-0.nightly-2019-04-18-210657
OLM version info:
io.openshift.build.commit.id=c718ec855bb26a111d66ba2ba193d30e54f7feb1

How reproducible:
always

Steps to Reproduce:
1. Install the `singleNamespace` etcd operator in project default.
2. Check its status, it works well:
mac:OCP-21082 jianzhang$ oc get csv -n default
NAME                        DISPLAY              VERSION   REPLACES              PHASE
etcdoperator.v0.9.4         etcd                 0.9.4     etcdoperator.v0.9.2   Succeeded

Its installMode as below:
mac:OCP-21082 jianzhang$ oc get csv -n default etcdoperator.v0.9.4  -o yaml
...
  installModes:
  - supported: true
    type: OwnNamespace
  - supported: true
    type: SingleNamespace
  - supported: false
    type: MultiNamespace
  - supported: false
    type: AllNamespaces

mac:OCP-21082 jianzhang$ oc get pods -n default
NAME                                             READY     STATUS    RESTARTS   AGE
etcd-operator-578cc84b7f-rpqtp                   3/3       Running   0          64m

3. Install the `cluster wide` etcd operator in the default project "openshift-operators".

4. Check the clusterwide etcd operator status.

Actual results:
The csv of the clusterwide-etcd-operator failed:
mac:OCP-21082 jianzhang$ oc get csv -n openshift-operators
NAME                              DISPLAY       VERSION             REPLACES                          PHASE
etcdoperator.v0.9.4-clusterwide   etcd          0.9.4-clusterwide   etcdoperator.v0.9.2-clusterwide   Failed

mac:OCP-21082 jianzhang$ oc describe csv etcdoperator.v0.9.4-clusterwide  -n openshift-operators
...
  Install Modes:
    Supported:  true
    Type:       OwnNamespace
    Supported:  false
    Type:       SingleNamespace
    Supported:  false
    Type:       MultiNamespace
    Supported:  true
    Type:       AllNamespaces
...
Status:
  Certs Last Updated:  <nil>
  Certs Rotate At:     <nil>
  Conditions:
    Last Transition Time:  2019-04-19T06:07:44Z
    Last Update Time:      2019-04-19T06:07:44Z
    Message:               intersecting operatorgroups provide the same apis
    Phase:                 Failed
    Reason:                InterOperatorGroupOwnerConflict
  Last Transition Time:    2019-04-19T06:07:44Z
  Last Update Time:        2019-04-19T06:07:44Z
  Message:                 intersecting operatorgroups provide the same apis
  Phase:                   Failed
  Reason:                  InterOperatorGroupOwnerConflict
Events:
  Type     Reason                           Age   From                        Message
  ----     ------                           ----  ----                        -------
  Warning  InterOperatorGroupOwnerConflict  40m   operator-lifecycle-manager  intersecting operatorgroups provide the same apis

Expected results:
The clusterwide etcd operator should be installed successfully.

Additional info:
The OperatorGroup in project default:
mac:OCP-21082 jianzhang$ oc get og -n default default-jwng4 -o yaml
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  annotations:
    olm.providedAPIs: Cluster.v1alpha1.clusterregistry.k8s.io,CouchbaseCluster.v1.couchbase.com,DNSEndpoint.v1alpha1.multiclusterdns.federation.k8s.io,Domain.v1alpha1.multiclusterdns.federation.k8s.io,EtcdBackup.v1beta2.etcd.database.coreos.com,EtcdCluster.v1beta2.etcd.database.coreos.com,EtcdRestore.v1beta2.etcd.database.coreos.com,FederatedCluster.v1alpha1.core.federation.k8s.io,FederatedServiceStatus.v1alpha1.core.federation.k8s.io,FederatedTypeConfig.v1alpha1.core.federation.k8s.io,FederationConfig.v1alpha1.core.federation.k8s.io,IngressDNSRecord.v1alpha1.multiclusterdns.federation.k8s.io,PropagatedVersion.v1alpha1.core.federation.k8s.io,ReplicaSchedulingPreference.v1alpha1.scheduling.federation.k8s.io,ServiceDNSRecord.v1alpha1.multiclusterdns.federation.k8s.io
  creationTimestamp: 2019-04-19T05:25:01Z
  generateName: default-
  generation: 2
  name: default-jwng4
  namespace: default
  resourceVersion: "58614"
  selfLink: /apis/operators.coreos.com/v1/namespaces/default/operatorgroups/default-jwng4
  uid: 7b36dd11-6263-11e9-9886-06248344a592
spec:
  serviceAccount:
    metadata:
      creationTimestamp: null
  targetNamespaces:
  - default
status:
  lastUpdated: 2019-04-19T05:25:02Z
  namespaces:
  - default

The OperatorGroup in project openshift-operators:
mac:OCP-21082 jianzhang$ oc get og -n openshift-operators global-operators -o yaml
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  annotations:
    olm.providedAPIs: Cockroachdb.v1alpha1.charts.helm.k8s.io,Kafka.v1alpha1.kafka.strimzi.io,KafkaConnect.v1alpha1.kafka.strimzi.io,KafkaConnectS2I.v1alpha1.kafka.strimzi.io,KafkaMirrorMaker.v1alpha1.kafka.strimzi.io,KafkaTopic.v1alpha1.kafka.strimzi.io,KafkaUser.v1alpha1.kafka.strimzi.io
  creationTimestamp: 2019-04-19T03:19:41Z
  generation: 2
  name: global-operators
  namespace: openshift-operators
  resourceVersion: "110821"
  selfLink: /apis/operators.coreos.com/v1/namespaces/openshift-operators/operatorgroups/global-operators
  uid: f87c0ed2-6251-11e9-8a0a-028455df8cba
spec:
  serviceAccount:
    metadata:
      creationTimestamp: null
status:
  lastUpdated: 2019-04-19T03:21:52Z
  namespaces:
  - ""

The relevent olm-operator logs:
mac:OCP-21082 jianzhang$ oc logs olm-operator-78dff998fd-lx8d4 |grep etcd|grep -i "error"
time="2019-04-19T05:39:31Z" level=info msg="operatorgroup incorrect" csv=etcdoperator.v0.9.4 error="<nil>" id=7OL2S namespace=default phase=
time="2019-04-19T06:07:44Z" level=info msg="operatorgroup incorrect" csv=etcdoperator.v0.9.4-clusterwide error="<nil>" id=JBWsC namespace=openshift-operators phase=
time="2019-04-19T06:07:44Z" level=warning msg="error cleaning up CSV deployment" csv=etcdoperator.v0.9.4-clusterwide deployment=etcd-operator err="deployments.apps \"etcd-operator\" not found" id=L/duT namespace=openshift-operators phase=

Comment 1 Evan Cordell 2019-04-22 12:22:05 UTC
This is the correct behavior for OLM. If we didn't prevent the installation of a clusterwide and namespaced operator, then both operators would fight for control in the namespace that the non-global operator is watching. The error message `intersecting operatorgroups provide the same apis` indicates this state.

Comment 2 Jian Zhang 2019-04-23 09:34:28 UTC
@Evan

Thanks for your information! I understand, but it is a little confusing.
For now, there are two choices(`singleNamespace` and `AllNamespaces`) for the users on the Web console.
If the user installed an operator with the `singleNamespace` way. And then, the `AllNamespace` way should be forbidden to use. vice versa.
Reopen it since I think we should add this restricts on the Web console. Correct me if I'm wrong.

Comment 3 Evan Cordell 2019-04-29 14:41:12 UTC
The UX issues you raise are reasonable concerns, but we won't be able to fix them for 4.1/4.2. We can revisit in the future.