Bug 2047565 - PVC snapshot creation is not successful
Summary: PVC snapshot creation is not successful
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: rook
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ODF 4.10.0
Assignee: Madhu Rajanna
QA Contact: Jilju Joy
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-28 05:24 UTC by Jilju Joy
Modified: 2023-08-09 17:03 UTC (History)
5 users (show)

Fixed In Version: 4.10.0-132
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-13 18:52:21 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github red-hat-storage rook pull 334 0 None Merged Sync from upstream 1.8 to downstream 4.10 2022-01-28 05:59:38 UTC
Github rook rook pull 9665 0 None Merged csi: bump csi snapshotter image to v5 2022-01-28 05:59:38 UTC
Red Hat Product Errata RHSA-2022:1372 0 None None None 2022-04-13 18:52:29 UTC

Description Jilju Joy 2022-01-28 05:24:33 UTC
Description of problem (please be detailed as possible and provide log
snippests):
The test cases given below failed because PVC snapshot cannot be created.

tests/manage/pv_services/pvc_snapshot/test_pvc_snapshot.py::TestPvcSnapshot::test_pvc_snapshot[CephBlockPool]

tests/manage/pv_services/pvc_snapshot/test_pvc_snapshot.py::TestPvcSnapshot::test_pvc_snapshot[CephFileSystem]


Error logs :


could not sync content "snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174": failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174: "snapshot controller failed to update snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174 on API server: volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174\" is forbidden: User \"system:serviceaccount:openshift-storage:rook-csi-rbd-provisioner-sa\" cannot patch resource \"volumesnapshotcontents\" in API group \"snapshot.storage.k8s.io\" at the cluster scope"
I0127 13:58:15.997577       1 snapshot_controller_base.go:167] Failed to sync content "snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174", will retry again: failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174: "snapshot controller failed to update snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174 on API server: volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174\" is forbidden: User \"system:serviceaccount:openshift-storage:rook-csi-rbd-provisioner-sa\" cannot patch resource \"volumesnapshotcontents\" in API group \"snapshot.storage.k8s.io\" at the cluster scope"
I0127 13:58:15.997641       1 event.go:285] Event(v1.ObjectReference{Kind:"VolumeSnapshotContent", Namespace:"", Name:"snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174", UID:"bcdc3616-5a4a-4495-80a0-af40b5b7269b", APIVersion:"snapshot.storage.k8s.io/v1", ResourceVersion:"100679", FieldPath:""}): type: 'Warning' reason: 'SnapshotCreationFailed' Failed to create snapshot: failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174: "snapshot controller failed to update snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174 on API server: volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-ff5d47dc-0a3c-4da2-8bf5-c97a560f2174\" is forbidden: User \"system:serviceaccount:openshift-storage:rook-csi-rbd-provisioner-sa\" cannot patch resource \"volumesnapshotcontents\" in API group \"snapshot.storage.k8s.io\" at the cluster scope"



must-gather  logs - https://ceph-downstream-jenkins-csb-storage.apps.ocp4.prod.psi.redhat.com/job/ocs-ci/802/artifact/logs/failed_testcase_ocs_logs_1643286773/
===========================================================

Version of all relevant components (if applicable):
OCP 4.10.0-0.nightly-2022-01-26-234447
ODF 4.10.0-125

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes, PVC snapshot creation failing

Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
2

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:
Yes, the feature was working in ODF 4.9 and the previous internal builds of 4.10

Steps to Reproduce:
1. Create PVC snapshot

OR 
Run these tests:
tests/manage/pv_services/pvc_snapshot/test_pvc_snapshot.py::TestPvcSnapshot::test_pvc_snapshot[CephBlockPool]

tests/manage/pv_services/pvc_snapshot/test_pvc_snapshot.py::TestPvcSnapshot::test_pvc_snapshot[CephFileSystem]



Actual results:
Snapshot creation is not succeeding

Expected results:
Snapshot creation should be success.

Additional info:

Comment 5 Jilju Joy 2022-02-01 13:14:43 UTC
These test cases passed in the acceptance run of the ODF build 4.10.0-132.
tests/manage/pv_services/pvc_snapshot/test_pvc_snapshot.py::TestPvcSnapshot::test_pvc_snapshot[CephBlockPool]

tests/manage/pv_services/pvc_snapshot/test_pvc_snapshot.py::TestPvcSnapshot::test_pvc_snapshot[CephFileSystem]

Marking this bug as verified.

Comment 10 errata-xmlrpc 2022-04-13 18:52:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.10.0 enhancement, security & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1372


Note You need to log in before you can comment on or make changes to this bug.