Hide Forgot
Description of problem: I'd like to install the Etcd-operator v0.6.1, but there are two problems: 1, I didn't find any entry for selecting the version of the operators on the Web console. 2, In the backend, for etcd-operator, there are three versions: etcdoperator.v0.9.2, etcdoperator.v0.9.0, etcdoperator.v0.6.1. I want to use the v0.6.1, so I create a subscription with manual approve, but no corresponding csv generated. Version-Release number of selected component (if applicable): OLM: io.openshift.build.commit.id=1f312481ae3641eea471abb792f9b056206e4cf4 How reproducible: always Steps to Reproduce: 1. Install the OCP 4.0 2. Create a subscription, like below: [core@ip-10-0-37-169 ~]$ cat etcdoperator.v0.6.1-sub.yaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: etcd-v0.6.1 namespace: openshift-operators spec: channel: alpha name: etcd installPlanApproval: Manual source: installed-community-global-operators sourceNamespace: openshift-operators startingCSV: etcdoperator.v0.6.1 Actual results: No pods of etcd operator v0.6.1 created. No csv generated. And, the CSV of the "installplan" display "etcdoperator.v0.9.2"! I think it's wrong. [core@ip-10-0-37-169 ~]$ oc get sub NAME PACKAGE SOURCE CHANNEL etcd-v0.6.1 etcd installed-community-global-operators alpha [core@ip-10-0-37-169 ~]$ oc get installplan NAME CSV SOURCE APPROVAL APPROVED install-l9f92 etcdoperator.v0.9.2 Manual false [core@ip-10-0-37-169 ~]$ oc get csv No resources found. Expected results: 1, should display the entry of selecting the version of an operator on Web console. 2, The etcd operator v0.6.1 pods should be generated successfully. Additional info:
@jiazha, Have you ever create RBAC? if you run etcd-operator in command line, it require to create RBAD at first, if create operator in web console, RBAC seems needn't be setup by manual.
Sudha, > is this still a defect? Yes, I think so. ge, > Have you ever create RBAC? What do you mean? You mean I have to create the RBAC policy of the etcdoperator.v0.6.1 manually? As far as I know, the OLM will install the RBAC policies automatically when installing operators. Anyway, for this bug, etcd-operator has three versions, but only the etcdoperator.v0.9.2 can be installed successfully via the OLM. Or am I missing something?
@jiazha, I tried etcdoperator.v0.6.1 on 4.0 env successfully, of course, we need to create RBAC at first. 1. Create RBAC at first, Create RBAC for etcdoperator, it's not special for any one etcd operator version(0.6.1, 0.9.0,0.9.2) ./lgtest ==>this is script I wrote for myself Creating role with ROLE_NAME=etcd-operator, NAMESPACE=default clusterrole.rbac.authorization.k8s.io/etcd-operator created Creating role binding with ROLE_NAME=etcd-operator, ROLE_BINDING_NAME=etcd-operator, NAMESPACE=default clusterrolebinding.rbac.authorization.k8s.io/etcd-operator created 2.$ oc create -f subscription-etcd-operator.yaml subscription.operators.coreos.com/etcd-v0.6.1 created $ cat subscription-etcd-operator.yaml ==>this is what you wanted apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: etcd-v0.6.1 namespace: default spec: channel: alpha name: etcd installPlanApproval: Manual source: installed-community-global-operators sourceNamespace: default startingCSV: etcdoperator.v0.6.1 $ oc get sub NAME PACKAGE SOURCE CHANNEL etcd-v0.6.1 etcd installed-community-global-operators alpha [core@ip-10-0-xxx lgtest]$ oc get csv NAME DISPLAY VERSION REPLACES PHASE clusterlogging.v0.0.1 Cluster Logging 0.0.1 Succeeded svcat.v0.1.34 Service Catalog 0.1.34 Succeeded [core@ip-10-xxx lgtest]$ oc get installplan No resources found. 3.$ oc create -f deployment.yaml deployment.extensions/etcd-operator created 4.$ oc create -f example-etcd-cluster.yaml etcdcluster.etcd.database.coreos.com/example-etcd-cluster created [core@ip-10-0-xxx lgtest]$ oc get pods NAME READY STATUS RESTARTS AGE etcd-operator-77c788c44b-8648w 1/1 Running 0 103s example-etcd-cluster-0000 1/1 Running 0 56s example-etcd-cluster-0001 1/1 Running 0 40s example-etcd-cluster-0002 1/1 Running 0 19s $ oc describe pods etcd-operator-77c788c44b-8648w|grep etcd-operator Name: etcd-operator-77c788c44b-8648w Labels: name=etcd-operator Controlled By: ReplicaSet/etcd-operator-77c788c44b etcd-operator: Image: quay.io/coreos/etcd-operator:v0.6.1 Image ID: quay.io/coreos/etcd-operator@sha256:bd944a211eaf8f31da5e6d69e8541e7cada8f16a9f7a5a570b22478997819943 etcd-operator MY_POD_NAME: etcd-operator-77c788c44b-8648w (v1:metadata.name) Normal Scheduled 36s default-scheduler Successfully assigned default/etcd-operator-77c788c44b-8648w to ip-10-0-130-9.us-east-2.compute.internal Normal Pulling 29s kubelet, ip-10-0-xxx.us-east-2.compute.internal pulling image "quay.io/coreos/etcd-operator:v0.6.1" Normal Pulled 25s kubelet, ip-10-0-xxx.us-east-2.compute.internal Successfully pulled image "quay.io/coreos/etcd-operator:v0.6.1" [core@ip-10-0-xxx lgtest]$ oc create -f example-etcd-cluster.yaml
@sponnaga i have a question is why we need to config RBAC at command line mode, by we needn't do it on web console?? and is there detailed steps for enable etcd-operator and etcd-cluster in web console, we run it on ocp 3.11, and it's too complex in 4.0 NextGen, could you or Evan help to explain the details to me? thx
Ge, Thank you for your information! But, actually, it was failed per comment 5. > $ oc get sub NAME PACKAGE SOURCE CHANNEL etcd-v0.6.1 etcd installed-community-global-operators alpha [core@ip-10-0-xxx lgtest]$ oc get csv NAME DISPLAY VERSION REPLACES PHASE clusterlogging.v0.0.1 Cluster Logging 0.0.1 Succeeded svcat.v0.1.34 Service Catalog 0.1.34 Succeeded [core@ip-10-xxx lgtest]$ oc get installplan No resources found. For step 2 in comment 5, no installplan/csv objects of the etcd-operator generated. It was failed. > 3.$ oc create -f deployment.yaml deployment.extensions/etcd-operator created For step 3 in comment 5, you were creating the etcd-operator by using the deployment YAML, not the OLM.
Ge, Jian, You shouldn't need to create RBAC for things installed via an OLM Subscription as long as the resolved CSV specifies its requirements properly. A Subscription is meant to keep an operator installation up-to-date in accordance with the latest CSV associated with the specified package and channel in the CatalogSource targeted by the Subscription. Creating a manual Subscription with a startingCSV will let you generate successive manual approval InstallPlans with CSVs leading up to the latest for the CatalogSource/package/channel. You could generate a CatalogSource that has a channel with the latest CSV pointing to the version of the CSV you want. This would let you generate an automatic Subscription - and when the CatalogSource is updated point to a newer CSV in that package/channel, your installation will be upgrade. > No pods of etcd operator v0.6.1 created. No csv generated. This should not be the expected behavior of a Subscription set to manual approval. You should expect an InstallPlan set to manual approval to be generated. This will not create any operator resources until it is "approved". > And, the CSV of the "installplan" display "etcdoperator.v0.9.2"! I think it's wrong. We merged a fix to respect starting CSV: https://github.com/operator-framework/operator-lifecycle-manager/pull/676 but you also need to make sure the startingCSV has a replaces chain to the latest (v0.9.2) in the CatalogSource you're using.
Nick, > You shouldn't need to create RBAC for things installed via an OLM Subscription as long as the resolved CSV specifies its requirements properly. Yes, thanks. Ge wasn't familiar with the OLM, I have told him this in comment 8. > This should not be the expected behavior of a Subscription set to manual approval. You should expect an InstallPlan set to manual approval to be generated. This will not create any operator resources until it is "approved". Yes, I know. In previous, we provide three versions for the etcd-operator in the ConfigMap. But, the user can only use the latest version(v0.9.2). And, we didn't provide the entry to select the version of the operator in Web console. And, even if we specify the low version(e.g. etcdoperator.v0.6.1) of the operator in the subscription YAML, still the latest version(etcdoperator.v0.9.2) installed after the manual approved. So, here the problem is that although we provide multi-versions for the operator in the source, the user can only use the latest version.
> And, even if we specify the low version(e.g. etcdoperator.v0.6.1) of the operator in the subscription YAML, still the latest version(etcdoperator.v0.9.2) installed after the manual approved. So, here the problem is that although we provide multi-versions for the operator in the source, the user can only use the latest version. Users can specify the _starting_ operator in the subscription. But the subscription's job is to update from that to newer versions. Can you please try the following: - Create a subscription for etcd, with starting CSV set to 0.6.1 and approval mode set to manual. - Show the installplans that are created in the namespace This will allow us to verify if there is a bug. We have an e2e test for this case: https://github.com/operator-framework/operator-lifecycle-manager/blob/4cd7c2e65e875241313b38e30b77249ac22e6066/test/e2e/subscription_e2e_test.go#L501
Evan, > Can you please try the following: > - Create a subscription for etcd, with starting CSV set to 0.6.1 and approval mode set to manual. > - Show the installplans that are created in the namespace Sure, I have tried to create the CatalogSourceConfig and v0.6.1 subscription objects, but no InstallPlan generated and got below errors: [jzhang@dhcp-140-18 multitenant]$ oc logs catalog-operator-686c4d88f6-kxcq8 |grep etcd time="2019-03-01T06:25:09Z" level=info msg="building connection to registry" currentSource="{etcd-jian openshift-operators}" id=OOPci source=etcd-jian time="2019-03-01T06:25:09Z" level=info msg="client hasn't yet become healthy, attempt a health check" currentSource="{etcd-jian openshift-operators}" id=OOPci source=etcd-jian time="2019-03-01T06:25:12Z" level=info msg="building connection to registry" currentSource="{etcd-jian openshift-operators}" id=PiLLn source=etcd-jian ... [jzhang@dhcp-140-18 multitenant]$ oc get sub NAME PACKAGE SOURCE CHANNEL etcd-v0.6.1 etcd etcd-jian alpha [jzhang@dhcp-140-18 multitenant]$ oc get ip No resources found. [jzhang@dhcp-140-18 multitenant]$ oc create -f csc.yaml catalogsourceconfig.marketplace.redhat.com/etcd-jian created [jzhang@dhcp-140-18 multitenant]$ oc create -f etcd-v0.6-sub.yaml subscription.operators.coreos.com/etcd-v0.6.1 created [jzhang@dhcp-140-18 multitenant]$ cat csc.yaml apiVersion: marketplace.redhat.com/v1alpha1 kind: CatalogSourceConfig metadata: name: etcd-jian namespace: openshift-marketplace spec: targetNamespace: openshift-operators packages: etcd [jzhang@dhcp-140-18 multitenant]$ cat etcd-v0.6-sub.yaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: etcd-v0.6.1 namespace: openshift-operators spec: channel: alpha name: etcd source: etcd-jian startingCSV: etcdoperator.v0.6.1 installPlanApproval: Manual So, how to specify the version of the etcd-operator for now? Cluster version is 4.0.0-0.nightly-2019-02-28-054829
Jian, Try adding "sourceNamespace: openshift-operators: to the Subscription spec.
Nick, Thanks! I will have a try.
Nick, > - Show the installplans that are created in the namespace As below: mac:aws-ocp jianzhang$ oc get sub NAME PACKAGE SOURCE CHANNEL camel-k camel-k installed-community-openshift-operators alpha cluster-logging cluster-logging installed-community-openshift-operators preview descheduler descheduler installed-community-openshift-operators alpha etcd-v0.6.1 etcd etcd-jian alpha federationv2 federationv2 installed-community-openshift-operators alpha mac:aws-ocp jianzhang$ oc get installplan NAME CSV SOURCE APPROVAL APPROVED install-b4pq6 descheduler.v0.0.1 Automatic true install-bs5gp federationv2.v0.0.2 Automatic true install-kl99t descheduler.v0.0.1 Automatic true install-rzpz7 descheduler.v0.0.1 Automatic true install-rzvmn clusterlogging.v0.0.1 Manual false Too many csv in this installPlan, I paste part of it. And, as you can see, the InstallPlan still to resolve the etcdoperator.v0.9.2. mac:aws-ocp jianzhang$ oc get installplan install-rzvmn -o yaml apiVersion: operators.coreos.com/v1alpha1 kind: InstallPlan metadata: creationTimestamp: 2019-03-04T10:40:30Z generateName: install- generation: 1 name: install-rzvmn namespace: openshift-operators ownerReferences: - apiVersion: operators.coreos.com/v1alpha1 blockOwnerDeletion: false controller: false kind: Subscription name: etcd-v0.6.1 uid: ea46b90f-3e69-11e9-b998-06210c98f03c - apiVersion: operators.coreos.com/v1alpha1 blockOwnerDeletion: false controller: false kind: Subscription name: descheduler uid: 3bc7a12e-3e49-11e9-a743-06210c98f03c - apiVersion: operators.coreos.com/v1alpha1 blockOwnerDeletion: false controller: false kind: Subscription name: cluster-logging uid: ff3dac8c-3e4e-11e9-b12e-020ce83edd9a - apiVersion: operators.coreos.com/v1alpha1 blockOwnerDeletion: false controller: false kind: Subscription name: federationv2 uid: 2e887707-3e49-11e9-a743-06210c98f03c - apiVersion: operators.coreos.com/v1alpha1 blockOwnerDeletion: false controller: false kind: Subscription name: camel-k uid: 1211b534-3e58-11e9-b1ae-06210c98f03c resourceVersion: "184864" selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/openshift-operators/installplans/install-rzvmn uid: ee31bfd6-3e69-11e9-b998-06210c98f03c spec: approval: Manual approved: false clusterServiceVersionNames: - clusterlogging.v0.0.1 - federationv2.v0.0.2 - camel-k-operator.v0.2.0 - etcdoperator.v0.9.2 - descheduler.v0.0.1 source: "" sourceNamespace: "" status: catalogSources: - installed-community-openshift-operators - etcd-jian phase: RequiresApproval plan: - resolving: clusterlogging.v0.0.1 resource: group: operators.coreos.com kind: ClusterServiceVersion manifest: | ...
I couldn't reproduce this exactly, but I did see that when specifying a startingCSV, multiple installplans are created (one for each version from startngCSV to current). Now that we can reproduce we will work on a fix.
This issue has been fixed, and I have a PR with an e2e test verifying the behavior: https://github.com/operator-framework/operator-lifecycle-manager/pull/794. The PR is not necessary to verify that this is fixed, though.
1, Create the CatalogSourceConfig in the openshift-marketplace project. Like below: [jzhang@dhcp-140-18 ocp43]$ cat csc.yaml apiVersion: operators.coreos.com/v1 kind: CatalogSourceConfig metadata: name: etcd-jian namespace: openshift-marketplace spec: targetNamespace: openshift-operators packages: etcd [jzhang@dhcp-140-18 ocp43]$ oc create -f csc.yaml catalogsourceconfig.operators.coreos.com/etcd-jian created 2, Create etcd subscription to specify the original version: "etcdoperator.v0.9.2-clusterwide", like below: [jzhang@dhcp-140-18 ocp43]$ cat sub.yaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: etcdoperator.v0.9.2-clusterwide namespace: openshift-operators spec: channel: clusterwide-alpha name: etcd source: etcd-jian startingCSV: etcdoperator.v0.9.2-clusterwide installPlanApproval: Manual sourceNamespace: openshift-operators [jzhang@dhcp-140-18 ocp43]$ oc create -f sub.yaml subscription.operators.coreos.com/etcdoperator.v0.9.2-clusterwide created 3, Check the csv in the "openshift-operators" project. Like below: [jzhang@dhcp-140-18 ocp43]$ oc get sub NAME PACKAGE SOURCE CHANNEL etcdoperator.v0.9.2-clusterwide etcd etcd-jian clusterwide-alpha [jzhang@dhcp-140-18 ocp43]$ oc get ip NAME CSV SOURCE APPROVAL APPROVED install-sllm2 etcdoperator.v0.9.2-clusterwide Manual false [jzhang@dhcp-140-18 ocp43]$ oc get csv No resources found. 4, Approve it and check it again. [jzhang@dhcp-140-18 ocp43]$ oc get ip (The user cannot find the Approval button to approve it on the Web console) NAME CSV SOURCE APPROVAL APPROVED install-rcjv2 etcdoperator.v0.9.4-clusterwide Manual false install-sllm2 etcdoperator.v0.9.2-clusterwide Manual true [jzhang@dhcp-140-18 ocp43]$ oc get csv NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.2-clusterwide etcd 0.9.2-clusterwide etcdoperator.v0.9.0 Succeeded [jzhang@dhcp-140-18 ocp43]$ oc get pods NAME READY STATUS RESTARTS AGE etcd-operator-6b57ff7bd-cf5ps 3/3 Running 0 7m40s 5, Approve the etcdoperator.v0.9.4-clusterwide and check it again. [jzhang@dhcp-140-18 ocp43]$ oc get csv NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.2-clusterwide etcd 0.9.2-clusterwide etcdoperator.v0.9.0 Replacing etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide etcdoperator.v0.9.2-clusterwide Installing [jzhang@dhcp-140-18 ocp43]$ oc get pods NAME READY STATUS RESTARTS AGE etcd-operator-7d86d767c-pq7gj 3/3 Running 0 55s [jzhang@dhcp-140-18 ocp43]$ oc get csv NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide etcdoperator.v0.9.2-clusterwide Succeeded LGTM, the upgrade works well on the back end. But, there are two issues remain: 1) the user cannot select the version of the operator on the Web console. 2) the Web console doesn't provide an entry to let the users approve the InstallPlan. Verify failed. OLM version: io.openshift.build.commit.id=9ba3512c5406b62179968e2432b284e9a30c321e io.openshift.build.commit.url=https://github.com/operator-framework/operator-lifecycle-manager/commit/9ba3512c5406b62179968e2432b284e9a30c321e
The marketplace UI intentionally hides this option. It is available via the OLM UI. The InstallPlan approval should be here: https://github.com/openshift/console/pull/1343
Evan, Thanks! But, where is the entry for the user to choose the operator's version on the Web console?
That's not currently an option via the UI
Evan Thanks for your information. But, now, it's difficult for the end user to use a specific version operator. I highly suggest we add this option on the Web console.
Evan, I'm sorry, but it would great if we can provide this option on the Web console from the users perspective. If we cannot do it in 4.1, we can move it to 4.2.
Marking FIXED. UI feature request will be created in JIRA.
Hi Stephen, Thanks! Verify this bug since we have a UI feature request in JIRA. Cluster version is 4.1.0-0.nightly-2019-04-28-064010 OLM version: io.openshift.build.commit.id=49ca4c57934ed4b0c974ce9bc3af354d5fc7146b
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758