Bug 1877338

Summary: Snapshot tests flake with "volumesnapshots.snapshot.storage.k8s.io "snapshot-rpk4n" not found"
Product: OpenShift Container Platform Reporter: Jan Safranek <jsafrane>
Component: StorageAssignee: Jan Safranek <jsafrane>
Storage sub component: Kubernetes QA Contact: Wei Duan <wduan>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: aos-bugs
Version: 4.6   
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:39:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jan Safranek 2020-09-09 12:19:04 UTC
AWS EBS CSI tests fail with this error:

# External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should delete the VolumeSnapshotContent according to its deletion policy
fail [k8s.io/kubernetes.0-rc.2/test/e2e/storage/testsuites/snapshottable.go:450]: Unexpected error:
    <*errors.StatusError | 0xc001752820>: {
        ErrStatus: {
            TypeMeta: {Kind: "Status", APIVersion: "v1"},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            Status: "Failure",
            Message: "volumesnapshots.snapshot.storage.k8s.io \"snapshot-wmb7p\" not found",
            Reason: "NotFound",
            Details: {
                Name: "snapshot-wmb7p",
                Group: "snapshot.storage.k8s.io",
                Kind: "volumesnapshots",
                UID: "",
                Causes: nil,
                RetryAfterSeconds: 0,
            Code: 404,
    volumesnapshots.snapshot.storage.k8s.io "snapshot-wmb7p" not found

Example: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_csi-external-provisioner/31/pull-ci-openshift-csi-external-provisioner-master-e2e-aws-csi/1303642383254032384

Full search: https://search.ci.openshift.org/?search=volumesnapshots.snapshot.storage.k8s.io+.*+not+found&maxAge=336h&context=15&type=junit&name=&maxMatches=5&maxBytes=20971520&groupBy=job

Comment 1 Jan Safranek 2020-09-09 13:06:43 UTC
Upstream PR: https://github.com/kubernetes/kubernetes/pull/94653

Comment 3 Jan Safranek 2020-09-16 15:38:44 UTC
We need to bump k8s/kubernetes in openshift/origin to fix the test.

Comment 4 Jan Safranek 2020-09-17 09:56:12 UTC
Fixed as part of https://github.com/openshift/origin/pull/25495

Comment 6 Wei Duan 2020-09-23 02:38:17 UTC
Checking the latest ci result, after PR merged, no failure case found due to "VolumeSnapshot not found" when cleaning up the volumesnapshot. 
Marked it as VERIFIED.

Comment 9 errata-xmlrpc 2020-10-27 16:39:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.