Bug 1917587

Summary: Manila CSI operator becomes degraded if user doesn't have permissions to list share types
Product: OpenShift Container Platform Reporter: Mike Fedosin <mfedosin>
Component: StorageAssignee: Mike Fedosin <mfedosin>
Storage sub component: OpenStack CSI Drivers QA Contact: Wei Duan <wduan>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: urgent CC: aos-bugs, atragler, bdobreli, eduen, igreen, m.andre, piqin, pprinett, tbarron
Version: 4.7   
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-24 15:54:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1918367    

Description Mike Fedosin 2021-01-18 20:18:52 UTC
Description of problem:
Now to detect whether Manila service is available in the cluster, the operator sends a request to fetch all share types. If the cloud returns ErrEndpointNotFound, the operator becomes Disabled and stops working, but if some other error is returned, then the operator changes its status to Degraded and prevents cluster upgrades.
The latter happens when the user doesn't have enough permissions to list share types and the system returns 403 error. We need to tolerate such errors and set the operator status to Disabled too.

How reproducible:

Steps to Reproduce:
1. Deploy an OpenStack with enabled Manila service
2. Forbid a user to list share types in Manila
3. Deploy an OpenShift cluster

Actual results:
Manila operator becomes Degraded and prevents upgrades

Expected results:
Manila operator should be Disabled in this case

Comment 14 Wei Duan 2021-02-05 01:28:35 UTC
I verified on our openstack platform:
1. Simulate to get 404 response when check the share type: 
$ manila type-list
/usr/lib/python3.6/site-packages/manilaclient/v1/contrib/list_extensions.py:22: UserWarning: Module manilaclient.v1.contrib.list_extensions is deprecated (taken as a basis for manilaclient.v2.contrib.list_extensions). The preferable way to get a client class or object is to use the manilaclient.client module.
  "Module manilaclient.v1.contrib.list_extensions is deprecated "
ERROR: Not Found (HTTP 404) (Request-ID: req-e5345caf-5549-4625-bb83-5c01841f7365)

2. Check the manila csi driver and CSO:
  $ oc get co storage -o yaml
  - lastTransitionTime: "2021-02-04T13:26:37Z"
    message: |-
      ManilaCSIDriverOperatorCRAvailable: CSI driver for Manila is disabled: Cannot find API to fetch Manila share types
      OpenStackCinderCSIDriverOperatorCRAvailable: All is well
    reason: AsExpected
    status: "True"
    type: Available

  $ oc get co storage
  NAME      VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
  storage   4.7.0-0.nightly-2021-02-03-165316   True        False         False      11h

So I change the status to VERIFIED

@igreen Feel free to add comments according your test result, thanks.

Comment 17 Wei Duan 2021-02-18 05:25:02 UTC
Hi @igreen,
See the 4.6 backport in https://bugzilla.redhat.com/show_bug.cgi?id=1918367, it plans to be shipped with 4.6.18.

Comment 20 errata-xmlrpc 2021-02-24 15:54:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Comment 21 Pierre Prinetti 2021-07-22 11:37:28 UTC
Needinfo answered already.