Bug 1902711
| Summary: | Tracker for Bug #1903078 Deleting VolumeSnapshotClass makes VolumeSnapshot not Ready | |||
|---|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat OpenShift Container Storage | Reporter: | Jilju Joy <jijoy> | |
| Component: | csi-driver | Assignee: | Humble Chirammal <hchiramm> | |
| Status: | CLOSED ERRATA | QA Contact: | Jilju Joy <jijoy> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 4.6 | CC: | asriram, dwalveka, edonnell, hchiramm, madam, mrajanna, muagarwa, nberry, ocs-bugs, tdesala | |
| Target Milestone: | --- | Keywords: | Tracking | |
| Target Release: | OCS 4.7.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: |
.You cannot create a PVC from a volume snapshot in the absence of `volumesnapshotclass`
A PVC can not be created from a volume snapshot in the absence of `volumesnapshotclass`.
This issue is caused because the status of the volume snapshot changes to a *not ready* state on deleting the `volumesnapshotclass`.
This issue has been fixed in OCP 4.7.0 and higher.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1903078 (view as bug list) | Environment: | ||
| Last Closed: | 2021-05-19 09:16:33 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1882359, 1903078, 1938134 | |||
This title "You cannot create PVC or clones from `snapshot`" should be changed. This title in release notes will give an impression that a PVC cannot be created from snapshot. This is applicable only in a situation where the snapshot class is not present. Why do we need to mention about clone here ? Changing the doc_text back from '+' to '?' as the content is changed. Erin, PTAL Thanks Jiliju/Madhu. Hi Erin I added the known issue statement as the current one is very misleading and people may think snapshot restore does not work at all You cannot restore PVC from `snapshot` to You cannot restore PVC from `snapshot` in absence of volume snapshotclass OCP BZ is already verified, moving this one to ON_QA Verified in version: OCP 4.7.0-0.nightly-2021-01-14-211319 ocs-operator.v4.7.0-230.ci List of Cephfs and RBD volume snapshots $ oc -n namespace-test-e9c5276f746b419fae7eeb463 get volumesnapshot NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE pvc-test-5b560b0d48a2447ebf69fdb91eda7a4-snapshot true pvc-test-5b560b0d48a2447ebf69fdb91eda7a4 3Gi cephfssnapshotclass snapcontent-86391466-211a-487f-aa27-141910a5f50d 70s 72s pvc-test-6aade12d84234b6f8e88649b0f9f41a-snapshot true pvc-test-6aade12d84234b6f8e88649b0f9f41a 3Gi rbdsnapshotclass snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 52s 53s Delete the volume snapshot classes $ oc delete volumesnapshotclass cephfssnapshotclass rbdsnapshotclass volumesnapshotclass.snapshot.storage.k8s.io "cephfssnapshotclass" deleted volumesnapshotclass.snapshot.storage.k8s.io "rbdsnapshotclass" deleted Verify the volume snapshotclasses are deleted $ oc get volumesnapshotclass cephfssnapshotclass rbdsnapshotclass Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io "cephfssnapshotclass" not found Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io "rbdsnapshotclass" not found List of Cephfs and RBD volume snapshots. Snapshots are in Ready state. $ oc -n namespace-test-e9c5276f746b419fae7eeb463 get volumesnapshot NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE pvc-test-5b560b0d48a2447ebf69fdb91eda7a4-snapshot true pvc-test-5b560b0d48a2447ebf69fdb91eda7a4 3Gi cephfssnapshotclass snapcontent-86391466-211a-487f-aa27-141910a5f50d 2m29s 2m31s pvc-test-6aade12d84234b6f8e88649b0f9f41a-snapshot true pvc-test-6aade12d84234b6f8e88649b0f9f41a 3Gi rbdsnapshotclass snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 2m11s 2m12s Verify the status of volume snapshot contents. Snapshot contents are in Ready state. $ oc get volumesnapshotcontent snapcontent-86391466-211a-487f-aa27-141910a5f50d snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT AGE snapcontent-86391466-211a-487f-aa27-141910a5f50d true 3221225472 Delete openshift-storage.cephfs.csi.ceph.com cephfssnapshotclass pvc-test-5b560b0d48a2447ebf69fdb91eda7a4-snapshot 3m21s snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 true 3221225472 Delete openshift-storage.rbd.csi.ceph.com rbdsnapshotclass pvc-test-6aade12d84234b6f8e88649b0f9f41a-snapshot 3m3s Verify volume snapshot class are still absent. $ oc get volumesnapshotclass cephfssnapshotclass rbdsnapshotclass Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io "cephfssnapshotclass" not found Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io "rbdsnapshotclass" not found (In reply to Jilju Joy from comment #13) > Verified in version: > OCP 4.7.0-0.nightly-2021-01-14-211319 > ocs-operator.v4.7.0-230.ci > > > List of Cephfs and RBD volume snapshots > $ oc -n namespace-test-e9c5276f746b419fae7eeb463 get volumesnapshot > NAME READYTOUSE SOURCEPVC > SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT > CREATIONTIME AGE > pvc-test-5b560b0d48a2447ebf69fdb91eda7a4-snapshot true > pvc-test-5b560b0d48a2447ebf69fdb91eda7a4 3Gi > cephfssnapshotclass snapcontent-86391466-211a-487f-aa27-141910a5f50d 70s > 72s > pvc-test-6aade12d84234b6f8e88649b0f9f41a-snapshot true > pvc-test-6aade12d84234b6f8e88649b0f9f41a 3Gi > rbdsnapshotclass snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 52s > 53s > > > Delete the volume snapshot classes > $ oc delete volumesnapshotclass cephfssnapshotclass rbdsnapshotclass > volumesnapshotclass.snapshot.storage.k8s.io "cephfssnapshotclass" deleted > volumesnapshotclass.snapshot.storage.k8s.io "rbdsnapshotclass" deleted > > > Verify the volume snapshotclasses are deleted > $ oc get volumesnapshotclass cephfssnapshotclass rbdsnapshotclass > Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io > "cephfssnapshotclass" not found > Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io > "rbdsnapshotclass" not found > > > List of Cephfs and RBD volume snapshots. Snapshots are in Ready state. > $ oc -n namespace-test-e9c5276f746b419fae7eeb463 get volumesnapshot > NAME READYTOUSE SOURCEPVC > SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT > CREATIONTIME AGE > pvc-test-5b560b0d48a2447ebf69fdb91eda7a4-snapshot true > pvc-test-5b560b0d48a2447ebf69fdb91eda7a4 3Gi > cephfssnapshotclass snapcontent-86391466-211a-487f-aa27-141910a5f50d > 2m29s 2m31s > pvc-test-6aade12d84234b6f8e88649b0f9f41a-snapshot true > pvc-test-6aade12d84234b6f8e88649b0f9f41a 3Gi > rbdsnapshotclass snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 > 2m11s 2m12s > > > Verify the status of volume snapshot contents. Snapshot contents are in > Ready state. > $ oc get volumesnapshotcontent > snapcontent-86391466-211a-487f-aa27-141910a5f50d > snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 > NAME READYTOUSE RESTORESIZE > DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS > VOLUMESNAPSHOT AGE > snapcontent-86391466-211a-487f-aa27-141910a5f50d true 3221225472 > Delete openshift-storage.cephfs.csi.ceph.com cephfssnapshotclass > pvc-test-5b560b0d48a2447ebf69fdb91eda7a4-snapshot 3m21s > snapcontent-83ba7737-569a-4049-a868-134bb6a1b637 true 3221225472 > Delete openshift-storage.rbd.csi.ceph.com rbdsnapshotclass > pvc-test-6aade12d84234b6f8e88649b0f9f41a-snapshot 3m3s > > > Verify volume snapshot class are still absent. > $ oc get volumesnapshotclass cephfssnapshotclass rbdsnapshotclass > Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io > "cephfssnapshotclass" not found > Error from server (NotFound): volumesnapshotclasses.snapshot.storage.k8s.io > "rbdsnapshotclass" not found Snapshot restore succeeded in the absence of volume snapshotclass. >> This issue is caused because the status of the volume snapshot changes to
> Error on deleting the volumesnapshotclass.
Disha, the status of volumesnapshot changes to "not ready" in this case and not really "error" state. so, that part of the text can be corrected. also, better avoid space between `volume` and `snapshot` above.
(In reply to Humble Chirammal from comment #20) > >> This issue is caused because the status of the volume snapshot changes to > > Error on deleting the volumesnapshotclass. > > Disha, the status of volumesnapshot changes to "not ready" in this case and > not really "error" state. so, that part of the text can be corrected. also, > better avoid space between `volume` and `snapshot` above. Thank you for the inputs, Humble. Changed to A PVC can not be created from a volume snapshot in the absence of volumesnapshotclass. This issue is caused because the status of the volume snapshot changes to a `not ready` state on deleting the volumesnapshotclass. To work around this issue, recreate the volumesnapshotclass. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat OpenShift Container Storage 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2041 |
Description of problem (please be detailed as possible and provide log snippests): Volume snapshot status in UI turns to 'Error' after deleting the volume snapshot class. In the yaml output, the value of 'readyToUse' parameter became false. Volume snapshot content remains in Ready state. $ oc describe volumesnapshot pvccephfs-snapshot -n test-snapshot-delete Name: pvccephfs-snapshot Namespace: test-snapshot-delete Labels: <none> Annotations: <none> API Version: snapshot.storage.k8s.io/v1beta1 Kind: VolumeSnapshot Metadata: Creation Timestamp: 2020-11-30T11:27:34Z Finalizers: snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection snapshot.storage.kubernetes.io/volumesnapshot-bound-protection Generation: 1 Managed Fields: API Version: snapshot.storage.k8s.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:spec: .: f:source: .: f:persistentVolumeClaimName: f:volumeSnapshotClassName: Manager: Mozilla Operation: Update Time: 2020-11-30T11:27:34Z API Version: snapshot.storage.k8s.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: f:status: .: f:boundVolumeSnapshotContentName: f:creationTime: f:error: .: f:message: f:time: f:readyToUse: f:restoreSize: Manager: snapshot-controller Operation: Update Time: 2020-11-30T11:33:24Z Resource Version: 174170 Self Link: /apis/snapshot.storage.k8s.io/v1beta1/namespaces/test-snapshot-delete/volumesnapshots/pvccephfs-snapshot UID: 6c38f430-3202-435b-b464-bb7f90476761 Spec: Source: Persistent Volume Claim Name: pvccephfs Volume Snapshot Class Name: ocs-storagecluster-cephfsplugin-snapclass Status: Bound Volume Snapshot Content Name: snapcontent-6c38f430-3202-435b-b464-bb7f90476761 Creation Time: 2020-11-30T11:27:36Z Error: Message: Failed to get snapshot class with error volumesnapshotclass.snapshot.storage.k8s.io "ocs-storagecluster-cephfsplugin-snapclass" not found Time: 2020-11-30T11:33:24Z Ready To Use: false Restore Size: 5Gi Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CreatingSnapshot 21m snapshot-controller Waiting for a snapshot test-snapshot-delete/pvccephfs-snapshot to be created by the CSI driver. Warning GetSnapshotClassFailed 15m snapshot-controller Failed to get snapshot class with error volumesnapshotclass.snapshot.storage.k8s.io "ocs-storagecluster-cephfsplugin-snapclass" not found $ oc describe volumesnapshot pvcrbd-snapshot -n test-snapshot-delete Name: pvcrbd-snapshot Namespace: test-snapshot-delete Labels: <none> Annotations: <none> API Version: snapshot.storage.k8s.io/v1beta1 Kind: VolumeSnapshot Metadata: Creation Timestamp: 2020-11-30T11:28:06Z Finalizers: snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection snapshot.storage.kubernetes.io/volumesnapshot-bound-protection Generation: 1 Managed Fields: API Version: snapshot.storage.k8s.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:spec: .: f:source: .: f:persistentVolumeClaimName: f:volumeSnapshotClassName: Manager: Mozilla Operation: Update Time: 2020-11-30T11:28:06Z API Version: snapshot.storage.k8s.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: f:status: .: f:boundVolumeSnapshotContentName: f:creationTime: f:error: .: f:message: f:time: f:readyToUse: f:restoreSize: Manager: snapshot-controller Operation: Update Time: 2020-11-30T11:33:24Z Resource Version: 174169 Self Link: /apis/snapshot.storage.k8s.io/v1beta1/namespaces/test-snapshot-delete/volumesnapshots/pvcrbd-snapshot UID: 6830107d-03cb-4547-9939-4e346c63f2fb Spec: Source: Persistent Volume Claim Name: pvcrbd Volume Snapshot Class Name: ocs-storagecluster-rbdplugin-snapclass Status: Bound Volume Snapshot Content Name: snapcontent-6830107d-03cb-4547-9939-4e346c63f2fb Creation Time: 2020-11-30T11:28:07Z Error: Message: Failed to get snapshot class with error volumesnapshotclass.snapshot.storage.k8s.io "ocs-storagecluster-rbdplugin-snapclass" not found Time: 2020-11-30T11:33:24Z Ready To Use: false Restore Size: 5Gi Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CreatingSnapshot 21m snapshot-controller Waiting for a snapshot test-snapshot-delete/pvcrbd-snapshot to be created by the CSI driver. Warning GetSnapshotClassFailed 16m snapshot-controller Failed to get snapshot class with error volumesnapshotclass.snapshot.storage.k8s.io "ocs-storagecluster-rbdplugin-snapclass" not found $ oc -n test-snapshot-delete get volumesnapshot NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE pvccephfs-snapshot false pvccephfs 5Gi ocs-storagecluster-cephfsplugin-snapclass snapcontent-6c38f430-3202-435b-b464-bb7f90476761 6m37s 6m39s pvcrbd-snapshot false pvcrbd 5Gi ocs-storagecluster-rbdplugin-snapclass snapcontent-6830107d-03cb-4547-9939-4e346c63f2fb 6m6s 6m7s $ oc get volumesnapshotcontent NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT AGE snapcontent-6830107d-03cb-4547-9939-4e346c63f2fb true 5368709120 Delete openshift-storage.rbd.csi.ceph.com ocs-storagecluster-rbdplugin-snapclass pvcrbd-snapshot 6m34s snapcontent-6c38f430-3202-435b-b464-bb7f90476761 true 5368709120 Delete openshift-storage.cephfs.csi.ceph.com ocs-storagecluster-cephfsplugin-snapclass pvccephfs-snapshot 7m6s $ oc get volumesnapshotclass No resources found Logs will be shared shortly. ========================================================================== Version of all relevant components (if applicable): ocs-operator.v4.6.0-160.ci Cluster version is 4.7.0-0.nightly-2020-11-29-133728 ceph version 14.2.8-111.el8cp (2e6029d57bc594eceba4751373da6505028c2650) nautilus (stable) rook: 4.6-73.15d47331.release_4.6 Snapshotter Version: v4.6.0-202011041933.p0-0-gef587a1-dirty CSI Driver version: release-4.6 and Git version: 9ae5f5cee7774048f65fdc48e07115e1c4f7b502 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Cannot delete volume snapshot Is there any workaround available to the best of your knowledge? Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 2 Can this issue reproducible? Yes Can this issue reproduce from the UI? Yes If this is a regression, please provide more details to justify this: Steps to Reproduce: 1. Create RBD or CephFs PVC and create a snapshot from it. Wait until the volume snapshot is Ready. 2. Delete the volume snapshot class used to create the above volume snapshot. (If using the default volume snapshot class, update reconcileStrategy of snapshotClasses to 'ignore' in the storagecluster to avoid reconciling ) 3. Check the status of volume snapshot. Actual results: Snapshot is not Ready Expected results: Volume snapshot should remain in Ready state. Additional info: