Bug 1908998
| Summary: | [cinder-csi-driver] doesn't detect the credentials change | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Wei Duan <wduan> |
| Component: | Storage | Assignee: | Mike Fedosin <mfedosin> |
| Storage sub component: | OpenStack CSI Drivers | QA Contact: | Wei Duan <wduan> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | high | ||
| Priority: | high | CC: | aos-bugs, lwan, mbooth, mfedosin, pprinett |
| Version: | 4.7 | Keywords: | UpcomingSprint |
| Target Milestone: | --- | ||
| Target Release: | 4.7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-02-24 15:46:32 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hello! The reason why I reassigned this bz is because we don't sync credentials directly, we just create a standard Credentials Request and expect that secrets will be synced across all namespaces automatically: https://github.com/openshift/cluster-storage-operator/blob/master/manifests/03_credentials_request_cinder.yaml Mike to clarify the bug, QE found that the storage operator appears to not be detecting a change in that credential, it appeared to continue to use the old one. We now need credentials to be able to be seamlessly rotated. All other operators tested were handling this well (detecting the change, restarting if necessary, etc), but the storage operator did not appear to do this. The CredentialsOperator does populate the Secrets, but the issue is it may need to update that Secret. Does this make sense? Verified pass on 4.7.0-0.nightly-2021-01-27-213348. I'd like to change the QA Contact but not assign, changed back. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633 |
Description of problem: After changing the credential with invalid password, cinder csi driver still works fine, and no related log found that it detected the secret change. Version-Release number of selected component (if applicable): 4.7.0-0.nightly-2020-12-14-080124 How reproducible: Always Steps to Reproduce: 1. Update the secret with invalid password: oc -n kube-system edit secret openstack-credentials ... 2. check the secret is reverted oc -n openshift-cluster-csi-drivers get secret openstack-cloud-credentials 3. Wait some time(10 minutes) and check driver log, no oc -n openshift-cluster-csi-drivers logs openstack-cinder-csi-driver-controller-68f4bccb58-6rbp7 -c csi-driver 4. create pod and pvc provisioned by csi driver still works $ oc get pod,pvc NAME READY STATUS RESTARTS AGE pod/mypod03 1/1 Running 0 14s NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/mypvc03 Bound pvc-6b100cb6-3fff-4bb7-b4fa-a451c3bae0c7 1Gi RWO standard-csi 15s Actual results: cinder-csi-driver doesn't detect the credentials changes, csi driver should not work after changing to invalid password Expected results: cinder-csi-driver should detect the credentials changes Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info: $ oc -n openshift-cluster-csi-drivers logs openstack-cinder-csi-driver-controller-68f4bccb58-6rbp7 -c csi-driver I1217 01:05:26.199744 1 driver.go:69] Driver: cinder.csi.openstack.org I1217 01:05:26.199869 1 driver.go:70] Driver version: 1.2.1@ I1217 01:05:26.199873 1 driver.go:71] CSI Spec version: 1.2.0 I1217 01:05:26.199881 1 driver.go:100] Enabling controller service capability: LIST_VOLUMES I1217 01:05:26.199886 1 driver.go:100] Enabling controller service capability: CREATE_DELETE_VOLUME I1217 01:05:26.199889 1 driver.go:100] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I1217 01:05:26.199892 1 driver.go:100] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I1217 01:05:26.199895 1 driver.go:100] Enabling controller service capability: LIST_SNAPSHOTS I1217 01:05:26.199898 1 driver.go:100] Enabling controller service capability: EXPAND_VOLUME I1217 01:05:26.199900 1 driver.go:100] Enabling controller service capability: CLONE_VOLUME I1217 01:05:26.199903 1 driver.go:100] Enabling controller service capability: LIST_VOLUMES_PUBLISHED_NODES I1217 01:05:26.199907 1 driver.go:112] Enabling volume access mode: SINGLE_NODE_WRITER I1217 01:05:26.199911 1 driver.go:122] Enabling node service capability: STAGE_UNSTAGE_VOLUME I1217 01:05:26.199914 1 driver.go:122] Enabling node service capability: EXPAND_VOLUME I1217 01:05:26.199917 1 driver.go:122] Enabling node service capability: GET_VOLUME_STATS I1217 01:05:26.200966 1 openstack.go:88] Block storage opts: {0 false false} I1217 01:05:26.289431 1 server.go:108] Listening for connections on address: &net.UnixAddr{Name:"/csi/csi.sock", Net:"unix"} $