Bug 1661484

Summary: The csv always state on the "succeeded" status even if the depend role are removed
Product: OpenShift Container Platform Reporter: Jian Zhang <jiazha>
Component: OLMAssignee: Evan Cordell <ecordell>
Status: CLOSED ERRATA QA Contact: Jian Zhang <jiazha>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.1.0CC: chezhang, dyan, jfan, jiazha, nhale, zitang
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:41:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jian Zhang 2018-12-21 10:49:54 UTC
Description of problem:
I removed the part of the role which the operator depends, the csv still state on the "SUCCEED" status. And, no error message report.

Version-Release number of selected component (if applicable):
mac:aws-ocp jianzhang$ oc exec olm-operator-86bb77db9b-r68ck -- olm -version
OLM version: 0.8.0
git commit: 61486be

How reproducible:
always

Steps to Reproduce:
1. Create an operatorgroup object in namespace "jian".
mac:aws-ocp jianzhang$ oc get operatorgroup -n jian
NAME                  AGE
couchbase-operators   4h

2. Create the etcd-operator in it.
mac:aws-ocp jianzhang$ oc get csv
NAME                        DISPLAY              VERSION   REPLACES              PHASE
etcdoperator.v0.9.2         etcd                 0.9.2     etcdoperator.v0.9.0   Succeeded

3. Remove below role from the etcd-operator role.
- apiGroups:
  - etcd.database.coreos.com
  resources:
  - etcdclusters
  - etcdbackups
  - etcdrestores
  verbs:
  - '*'

wait for some time

Actual results:
The csv still on "SUCCEED" status, not report error message.
mac:aws-ocp jianzhang$ oc get csv etcdoperator.v0.9.2 -o yaml 
...
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["*"],"apiGroups":["etcd.database.coreos.com"],"resources":["etcdclusters","etcdbackups","etcdrestores"]}
      status: Satisfied

mac:aws-ocp jianzhang$ oc describe role etcdoperator.v0.9.2-0 
Name:         etcdoperator.v0.9.2-0
Labels:       olm.owner=etcdoperator.v0.9.2
              olm.owner.namespace=jian
Annotations:  <none>
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  endpoints               []                 []              [*]
  events                  []                 []              [*]
  persistentvolumeclaims  []                 []              [*]
  pods                    []                 []              [*]
  services                []                 []              [*]
  deployments.apps        []                 []              [*]
  secrets                 []                 []              [get]

Expected results:
The csv should report the "NotSatisfied" message.

Additional info:
mac:aws-ocp jianzhang$ oc delete sa etcd-operator 
serviceaccount "etcd-operator" deleted
I also removed the depend ServiceAccount, it worked as expected.
mac:aws-ocp jianzhang$ oc get csv
NAME                        DISPLAY              VERSION   REPLACES              PHASE
etcdoperator.v0.9.2         etcd                 0.9.2     etcdoperator.v0.9.0   Pending

Comment 1 Evan Cordell 2018-12-21 13:19:52 UTC
Could we do one additional check to verify this bug? 


 oc -as <etcd-operater-serviceaccount-name> policy can-i get etcdclusters


I ask this just because our verification of PolicyRules is just that the serviceaccount has _some_ role bound to it with permission, not a specific role, so if the servieaccount still has access to etcdclusters through another binding, OLM won't change the CSV state. 

(it seems likely that this is a real regression, I just want to verify)

Comment 2 Jian Zhang 2018-12-26 12:24:39 UTC
Evan,

Thanks! Glad to know that. I tried it with that command and seems like the serviceaccount cannot access to etcdclusters.
Details as below:

Current version:
mac:aws-ocp jianzhang$ oc exec olm-operator-dff8b559d-6bfqt -- olm -version
OLM version: 0.8.0
git commit: 66be86a

mac:aws-ocp jianzhang$ oc get sa
NAME            SECRETS   AGE
builder         2         12m
default         2         12m
deployer        2         12m
etcd-operator   2         10m

mac:aws-ocp jianzhang$ oc --as etcd-operator policy can-i get etcdclusters
Command "can-i" is deprecated, use 'oc auth can-i'
no
mac:aws-ocp jianzhang$ oc --as etcd-operator policy can-i list etcdclusters
Command "can-i" is deprecated, use 'oc auth can-i'
no
mac:aws-ocp jianzhang$ oc --as etcd-operator policy can-i create etcdclusters
Command "can-i" is deprecated, use 'oc auth can-i'
no
mac:aws-ocp jianzhang$ oc --as etcd-operator policy can-i watch etcdclusters
Command "can-i" is deprecated, use 'oc auth can-i'
no
mac:aws-ocp jianzhang$ oc --as etcd-operator policy can-i update etcdclusters
Command "can-i" is deprecated, use 'oc auth can-i'
no


mac:aws-ocp jianzhang$ oc describe csv etcdoperator.v0.9.2
...
    Dependents:
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  namespaced rule:{"verbs":["*"],"apiGroups":["etcd.database.coreos.com"],"resources":["etcdclusters","etcdbackups","etcdrestores"]}
      Status:   Satisfied
      ...
Events:
  Type    Reason               Age   From                        Message
  ----    ------               ----  ----                        -------
  Normal  RequirementsUnknown  11m   operator-lifecycle-manager  requirements not yet checked
  Normal  AllRequirementsMet   11m   operator-lifecycle-manager  all requirements found, attempting install
  Normal  InstallSucceeded     11m   operator-lifecycle-manager  waiting for install components to report healthy

mac:aws-ocp jianzhang$ oc describe role etcdoperator.v0.9.2-0
Name:         etcdoperator.v0.9.2-0
Labels:       olm.owner=etcdoperator.v0.9.2
              olm.owner.namespace=jian
Annotations:  <none>
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  endpoints               []                 []              [*]
  events                  []                 []              [*]
  persistentvolumeclaims  []                 []              [*]
  pods                    []                 []              [*]
  services                []                 []              [*]
  deployments.apps        []                 []              [*]
  secrets                 []                 []              [get]

Comment 3 Nick Hale 2019-01-21 14:32:25 UTC
Jian,

Just to remove all doubt, I'm wondering if the `oc --as etcd-operator policy can-i get etcdclusters` needs to use the fully qualified name of the serviceaccount in question.

   oc --as system:serviceaccounts:<project>:etcd-operator policy can-i ...

I'm also wondering if the command is checking if the serviceaccount can perform the action on all namespaces, default, global etc... (Do we need to namespace the command with a `-n <namespace>`)

One last thing - Could you try using the non-deprecated `oc auth can-i` instead?

Thank you!

Comment 5 Jian Zhang 2019-01-25 11:31:59 UTC
LGTM, verify it. Details as below:

OLM version:
[jzhang@dhcp-140-18 ~]$ oc image info registry.svc.ci.openshift.org/ocp/4.0-art-latest-2019-01-24-184525@sha256:7d11b08fa707267d073ca3ccf84a72efdac671044bced9e88ee90515c64168fe|grep commit.id
               io.openshift.build.commit.id=85363e8ec624df90ce679ff26948fc8d8572bcf5


I remove the below resource from both the role and clusterrole, as below:

- apiGroups:
  - etcd.database.coreos.com
  resources:
  - etcdclusters
  - etcdbackups
  - etcdrestores
  verbs:
  - '*'

mac:origin-22 jianzhang$ oc describe role etcdoperator.v0.9.2-0
Name:         etcdoperator.v0.9.2-0
Labels:       olm.owner=etcdoperator.v0.9.2
              olm.owner.namespace=openshift-operators
Annotations:  <none>
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  endpoints               []                 []              [*]
  events                  []                 []              [*]
  persistentvolumeclaims  []                 []              [*]
  pods                    []                 []              [*]
  services                []                 []              [*]
  deployments.apps        []                 []              [*]
  secrets                 []                 []              [get]

mac:origin-22 jianzhang$ oc describe clusterrole etcdoperator.v0.9.2-0
Name:         etcdoperator.v0.9.2-0
Labels:       olm.owner=etcdoperator.v0.9.2
              olm.owner.namespace=openshift-operators
Annotations:  <none>
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  endpoints               []                 []              [*]
  events                  []                 []              [*]
  persistentvolumeclaims  []                 []              [*]
  pods                    []                 []              [*]
  services                []                 []              [*]
  deployments.apps        []                 []              [*]
  secrets                 []                 []              [get]


mac:origin-22 jianzhang$ oc --as system:serviceaccount:openshift-operators:etcd-operator policy can-i create etcdclusters
Command "can-i" is deprecated, use 'oc auth can-i'
no
mac:origin-22 jianzhang$ oc get csv
NAME                  DISPLAY   VERSION   REPLACES              PHASE
etcdoperator.v0.9.2   etcd      0.9.2     etcdoperator.v0.9.0   Pending
mac:origin-22 jianzhang$ oc describe csv etcdoperator.v0.9.2
...
Events:
  Type    Reason               Age                From                        Message
  ----    ------               ----               ----                        -------
  Normal  RequirementsUnknown  57m                operator-lifecycle-manager  requirements not yet checked
  Normal  AllRequirementsMet   57m                operator-lifecycle-manager  all requirements found, attempting install
  Normal  InstallSucceeded     57m                operator-lifecycle-manager  waiting for install components to report healthy
  Normal  RequirementsNotMet   10s (x27 over 5m)  operator-lifecycle-manager  one or more requirements couldn't be found

Comment 8 errata-xmlrpc 2019-06-04 10:41:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758