Bug 1909055 - Can't take a snapshot for a pre provisioned volumesnapshotcontent instance
Summary: Can't take a snapshot for a pre provisioned volumesnapshotcontent instance
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.7
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.7.0
Assignee: Christian Huffman
QA Contact: Qin Ping
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-18 09:08 UTC by Qin Ping
Modified: 2021-01-06 15:08 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-01-06 15:08:20 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Qin Ping 2020-12-18 09:08:53 UTC
Description of problem:
Can't take a snapshot for a pre provisioned volumesnapshotcontent instance

Version-Release number of selected component (if applicable):
4.7.0-0.nightly-2020-12-17-201522

How reproducible:
Always

Steps to Reproduce:
1. Create a volumesnapshotcontent object with:
$ cat vsc-manual.yaml 
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotContent
metadata:
  name: test-2
spec:
  deletionPolicy: Retain
  driver: ebs.csi.aws.com
  source:
    volumeHandle: vol-0613e88470c9a9b9d
  volumeSnapshotClassName: csi-snapshotclass
  volumeSnapshotRef:
    apiVersion: snapshot.storage.k8s.io/v1beta1
    kind: VolumeSnapshot
    name: mysnapshot-v2
    namespace: piqin
2. Create a volumesnapshot object with:
$ cat ss-manual.yaml 
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: mysnapshot-v2
spec:
  volumeSnapshotClassName: csi-snapshotclass
  source:
    volumeSnapshotContentName: test-2


Actual results:
$ oc get volumesnapshot
NAME            READYTOUSE   SOURCEPVC    SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS       SNAPSHOTCONTENT                                    CREATIONTIME   AGE
mysnapshot-v2   false                     test-2                                csi-snapshotclass                                                                     12m

$ oc get volumesnapshotcontent test-2
NAME     READYTOUSE   RESTORESIZE   DELETIONPOLICY   DRIVER            VOLUMESNAPSHOTCLASS   VOLUMESNAPSHOT   AGE
test-2   false                      Retain           ebs.csi.aws.com   csi-snapshotclass     mysnapshot-v2    14m

volumesnapshotcontent/test-2 is not ready to use for a long time.

Get some events:
Warning  SnapshotCreationFailed               12m   csi-snapshotter ebs.csi.aws.com  Failed to create snapshot: failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
  Warning  SnapshotContentCheckandUpdateFailed  12m   csi-snapshotter ebs.csi.aws.com  Failed to check and update snapshot content: failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"


Expected results:
After the volumesnapshot/mysnapshot-v2 is created the volumesnapshotcontent/test-2 should be in boud status, and snapshot should be taken for the volume defined in the volumesnapshotcontent/test-2.

Additional info:
logs get from csi driver:
E1218 08:35:54.102915       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:36:01.743382       1 snapshot_controller.go:122] checkandUpdateContentStatus [test-2]: error occurred failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:36:01.743391       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:36:54.103018       1 snapshot_controller.go:122] checkandUpdateContentStatus [test-2]: error occurred failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:36:54.103027       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:37:54.103049       1 snapshot_controller.go:122] checkandUpdateContentStatus [test-2]: error occurred failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:37:54.103059       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:38:54.103112       1 snapshot_controller.go:122] checkandUpdateContentStatus [test-2]: error occurred failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:38:54.103120       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:39:54.103199       1 snapshot_controller.go:122] checkandUpdateContentStatus [test-2]: error occurred failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:39:54.103209       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:40:54.103320       1 snapshot_controller.go:122] checkandUpdateContentStatus [test-2]: error occurred failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"
E1218 08:40:54.103328       1 snapshot_controller_base.go:261] could not sync content "test-2": failed to take snapshot of the volume, vol-0613e88470c9a9b9d: "cannot create snapshot. Snapshot content test-2 not bound to a snapshot"

Comment 3 Christian Huffman 2021-01-06 15:08:20 UTC
> I thought when we set spec.source.volumeHandle, then the snapshot will be token for the volume, when the volumesnapshotcontent is bound to a volumesnapshot.

We use these values to indicate if the content was dynamically provisioned or should represent a pre-existing snapshot on the backend data store. Only one should ever be used. In the case of dynamic provisioning, then we automatically populate the volumeHandle. In the case of static provisioning (this discussion), then we should use snapshotHandle.

In this case we need to create the Kubernetes representation of the backend snapshot (that already exists), so we're manually creating the VolumeSnapshotContent with a snapshotHandle to point to the backing snapshot. No additional snapshot is created on the backend.

When we specify the volumeHandle (indicating that this is a dynamically provisioned VolumeSnapshotContent that corresponds to a VolumeSnapshot of a PVC), then we need to create it on the storage backend. 

Both volumeHandle and snapshotHandle are a reference to the ID of the snapshot on the backend, but volumeHandle indicates that we dynamically created it, and snapshotHandle indicates that this was a pre-provisioned snapshot.

Closing this as NOTABUG, as this issue was caused by configuration.


Note You need to log in before you can comment on or make changes to this bug.