Bug 2101383
| Summary: | Latest OCP 4.11 builds break OADP 1.0.3 CSI functionality | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | mperetz <mperetz> |
| Component: | Storage | Assignee: | OpenShift Storage Bugzilla Bot <ocp-storage-bot> |
| Storage sub component: | Storage | QA Contact: | Wei Duan <wduan> |
| Status: | CLOSED WONTFIX | Docs Contact: | |
| Severity: | unspecified | ||
| Priority: | unspecified | CC: | akarol, dagur, dymurray, ernelson, jsafrane, mberube, pvauter, rhaigner, vkolli |
| Version: | 4.11 | ||
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-06-29 06:36:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I doubt Velero uses CSIStorageCapacity. It complains about "snapshot.storage.k8s.io/v1beta1" and that has been indeed removed in 4.11, as snapshot.storage.k8s.io/v1 is available since OCP 4.7 and v1beta1 was used as tech preview. Why can't you switch to v1? Jan is correct that the issue is the 1.0.3 CSI plugin expected v1beta1 to exist and fails now that it's removed. For OADP 1.1 we are using the latest CSI plugin which uses `v1`. If we want 1.0.x to support 4.11 we'll need to backport this fix to oadp-1.0. Do you want to reassign the BZ to yourself? Otherwise I will close it, as there is nothing to fix on OCP side. Closed as won't fix and opened a bug on OADP Jira project instead: https://issues.redhat.com/browse/OADP-585 |
Description of problem: On latest OCP 4.11 builds CSIStorageCapacity API version changed to v1 instead of v1beta1. This is probably breaks the CSI functionality on OADP 1.0.3 release, as OADP 1.0.3 the velero searches for v1beta1. On ocp-version:4.11.0-0.nightly-2022-06-15-222801 the issue didn't occur, but on latest build the velero pod panics with this error message: msg="The 'EnableCSI' feature flag was specified, but CSI API group [snapshot.storage.k8s.io/v1beta1] was not found. and seems like the only CRD with v1beta1 from this API group was CSIStorageCapacity. For example, I checked on an OSP cluster with 4.11.0-0.nightly-2022-05-25-193227 and I saw that it has: (venv) [mperetz@mperetz oadp-qe-automation]$ oc api-resources --api-group=storage.k8s.io NAME SHORTNAMES APIVERSION NAMESPACED KIND csidrivers storage.k8s.io/v1 false CSIDriver csinodes storage.k8s.io/v1 false CSINode csistoragecapacities storage.k8s.io/v1beta1 true CSIStorageCapacity <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< storageclasses sc storage.k8s.io/v1 false StorageClass volumeattachments storage.k8s.io/v1 false VolumeAttachment (venv) [mperetz@mperetz oadp-qe-automation]$ while ocp-version:4.11.0-0.nightly-2022-06-15-222801 has (venv) [mperetz@mperetz oadp-qe-automation]$ oc api-resources --api-group=storage.k8s.io NAME SHORTNAMES APIVERSION NAMESPACED KIND csidrivers storage.k8s.io/v1 false CSIDriver csinodes storage.k8s.io/v1 false CSINode csistoragecapacities storage.k8s.io/v1 true CSIStorageCapacity <<<<<<<<<<<<<<<<<<<<<<<<<<<< storageclasses sc storage.k8s.io/v1 false StorageClass volumeattachments storage.k8s.io/v1 false VolumeAttachment Version-Release number of selected component (if applicable): latest OCP 4.11 build (last checked: 4.11.0-0.nightly-2022-06-15-222801) and OADP 1.0.3 operator installed from production sources. How reproducible: 100% Steps to Reproduce: 1. On latest OCP 4.11, deploy OADP 1.0.3 from production sources. 2. After the OADP operator is installed the the controller pod is ready, create the following DPA CR (with CSI plugin): apiVersion: oadp.openshift.io/v1alpha1 kind: DataProtectionApplication metadata: namespace: openshift-adp name: example-velero spec: backupImages: true configuration: velero: defaultPlugins: - openshift - csi - aws noDefaultBackupLocation: true restic: enable: true 3. note that after a while the velero pods moves into crashloop, and this error appears on the velero pod logs: msg="The 'EnableCSI' feature flag was specified, but CSI API group [snapshot.storage.k8s.io/v1beta1] was not found. Actual results: Expected results: Master Log: Node Log (of failed PODs): msg="The 'EnableCSI' feature flag was specified, but CSI API group [snapshot.storage.k8s.io/v1beta1] was not found. PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info: