Bug 1877338 - Snapshot tests flake with "volumesnapshots.snapshot.storage.k8s.io "snapshot-rpk4n" not found"
Summary: Snapshot tests flake with "volumesnapshots.snapshot.storage.k8s.io "snapshot-...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.6
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.6.0
Assignee: Jan Safranek
QA Contact: Wei Duan
Depends On:
TreeView+ depends on / blocked
Reported: 2020-09-09 12:19 UTC by Jan Safranek
Modified: 2020-09-23 02:38 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed:
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Github openshift kubernetes pull 340 None closed Bug 1877338: UPSTREAM: 94653: Tolerate NotFound errors when deleting snapshots 2020-09-19 14:24:16 UTC
Github openshift origin pull 25495 None closed Bug 1873043: Bump to kube 1.19.0 2020-09-23 02:21:32 UTC

Description Jan Safranek 2020-09-09 12:19:04 UTC
AWS EBS CSI tests fail with this error:

# External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should delete the VolumeSnapshotContent according to its deletion policy
fail [k8s.io/kubernetes@v1.19.0-rc.2/test/e2e/storage/testsuites/snapshottable.go:450]: Unexpected error:
    <*errors.StatusError | 0xc001752820>: {
        ErrStatus: {
            TypeMeta: {Kind: "Status", APIVersion: "v1"},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            Status: "Failure",
            Message: "volumesnapshots.snapshot.storage.k8s.io \"snapshot-wmb7p\" not found",
            Reason: "NotFound",
            Details: {
                Name: "snapshot-wmb7p",
                Group: "snapshot.storage.k8s.io",
                Kind: "volumesnapshots",
                UID: "",
                Causes: nil,
                RetryAfterSeconds: 0,
            Code: 404,
    volumesnapshots.snapshot.storage.k8s.io "snapshot-wmb7p" not found

Example: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_csi-external-provisioner/31/pull-ci-openshift-csi-external-provisioner-master-e2e-aws-csi/1303642383254032384

Full search: https://search.ci.openshift.org/?search=volumesnapshots.snapshot.storage.k8s.io+.*+not+found&maxAge=336h&context=15&type=junit&name=&maxMatches=5&maxBytes=20971520&groupBy=job

Comment 1 Jan Safranek 2020-09-09 13:06:43 UTC
Upstream PR: https://github.com/kubernetes/kubernetes/pull/94653

Comment 3 Jan Safranek 2020-09-16 15:38:44 UTC
We need to bump k8s/kubernetes in openshift/origin to fix the test.

Comment 4 Jan Safranek 2020-09-17 09:56:12 UTC
Fixed as part of https://github.com/openshift/origin/pull/25495

Comment 6 Wei Duan 2020-09-23 02:38:17 UTC
Checking the latest ci result, after PR merged, no failure case found due to "VolumeSnapshot not found" when cleaning up the volumesnapshot. 
Marked it as VERIFIED.

Note You need to log in before you can comment on or make changes to this bug.