Bug 1949357
| Summary: | manila-csi-controller pod not running due to secret lack(in another ns) | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Wei Duan <wduan> |
| Component: | Storage | Assignee: | Jan Safranek <jsafrane> |
| Storage sub component: | Operators | QA Contact: | Wei Duan <wduan> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | high | ||
| Priority: | high | CC: | aos-bugs, cahl, dollierp, fdeutsch, jsafrane, mbooth, mfedosin, rlobillo |
| Version: | 4.8 | Keywords: | Regression |
| Target Milestone: | --- | ||
| Target Release: | 4.8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-07-27 23:00:24 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Wei Duan
2021-04-14 05:42:55 UTC
@Jan Maybe need your help on this but not stack team? This is some kind of block for our CI/upgrade due to the CSO status. Hi, I confirm this issue. Deploying OCP-4.8.0-0.nightly-2021-04-13-171608 on RHOS-PSI is failing for me with the same error. Regards. Denis. It's still not working with OCP 4.8.0-0.nightly-2021-04-17-044339.
Same issue:
> MountVolume.SetUp failed for volume "metrics-serving-cert" : secret "manila-csi-driver-controller-metrics-serving-cert" not found
@mfedosin is reporting that this is not fixed by https://github.com/openshift/csi-driver-manila-operator/pull/96 Yes, this is what I have: The fix #96 has been applied, and now I see a service monitor in the right namespace ❯ oc get servicemonitor -n openshift-manila-csi-driver NAME AGE manila-csi-driver-controller-monitor 3h22m But there is still no secret ❯ oc get secret -n openshift-manila-csi-driver manila-csi-driver-controller-metrics-serving-cert Error from server (NotFound): secrets "manila-csi-driver-controller-metrics-serving-cert" not found And Manila controller fails to start Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> Successfully assigned openshift-manila-csi-driver/openstack-manila-csi-controllerplugin-bc4b9cbb9-dd6xf to mfedosin-pnqxg-master-0 Warning FailedMount 96m (x7 over 155m) kubelet, mfedosin-pnqxg-master-0 Unable to attach or mount volumes: unmounted volumes=[metrics-serving-cert], unattached volumes=[metrics-serving-cert socket-dir cacert manila-csi-driver-controller-sa-token-lgb7n]: timed out waiting for the condition Warning FailedMount 35m (x19 over 151m) kubelet, mfedosin-pnqxg-master-0 Unable to attach or mount volumes: unmounted volumes=[metrics-serving-cert], unattached volumes=[socket-dir cacert manila-csi-driver-controller-sa-token-lgb7n metrics-serving-cert]: timed out waiting for the condition Warning FailedMount 22m (x15 over 139m) kubelet, mfedosin-pnqxg-master-0 Unable to attach or mount volumes: unmounted volumes=[metrics-serving-cert], unattached volumes=[manila-csi-driver-controller-sa-token-lgb7n metrics-serving-cert socket-dir cacert]: timed out waiting for the condition Warning FailedMount 7m10s (x82 over 157m) kubelet, mfedosin-pnqxg-master-0 MountVolume.SetUp failed for volume "metrics-serving-cert" : secret "manila-csi-driver-controller-metrics-serving-cert" not found Warning FailedMount 112s (x14 over 148m) kubelet, mfedosin-pnqxg-master-0 Unable to attach or mount volumes: unmounted volumes=[metrics-serving-cert], unattached volumes=[cacert manila-csi-driver-controller-sa-token-lgb7n metrics-serving-cert socket-dir]: timed out waiting for the condition One step further:
E0419 17:57:57.765388 1 base_controller.go:253] StaticResourceController reconciliation failed: ["service.yaml" (string): services "manila-csi-driver-controller-metrics" is forbidden: User "system:serviceaccount:openshift-cluster-csi-drivers:manila-csi-driver-operator" cannot get resource "services" in API group "" in the namespace "openshift-manila-csi-driver", "rbac/prometheus_role.yaml" (string): roles.rbac.authorization.k8s.io "manila-csi-driver-prometheus" is forbidden: user "system:serviceaccount:openshift-cluster-csi-drivers:manila-csi-driver-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:openshift-cluster-csi-drivers" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["endpoints"], Verbs:["get" "list" "watch"]}
{APIGroups:[""], Resources:["pods"], Verbs:["get" "list" "watch"]}
{APIGroups:[""], Resources:["services"], Verbs:["get" "list" "watch"]}, "rbac/prometheus_rolebinding.yaml" (string): roles.rbac.authorization.k8s.io "manila-csi-driver-prometheus" not found]
Verified pass on 4.8.0-0.nightly-2021-04-20-101404 *** Bug 1952144 has been marked as a duplicate of this bug. *** *** Bug 1954010 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |