Bug 1967175 - [IBM Z]: Pvc deletion failed to release claim to PV for local volume
Summary: [IBM Z]: Pvc deletion failed to release claim to PV for local volume
Keywords:
Status: CLOSED DUPLICATE of bug 1952820
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.8
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Jan Safranek
QA Contact: Wei Duan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-06-02 14:44 UTC by Sravika
Modified: 2021-07-27 08:01 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-08 08:38:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Must Gather Logs (10.61 MB, application/gzip)
2021-06-02 14:44 UTC, Sravika
no flags Details
Rook Operator log (266.23 KB, text/plain)
2021-06-02 14:45 UTC, Sravika
no flags Details

Description Sravika 2021-06-02 14:44:04 UTC
Created attachment 1788713 [details]
Must Gather Logs

Description of problem (please be detailed as possible and provide log
snippests):

During the uninstallation of OCS, delete the storage cluster and wait for the removal of the associated objects. After the storage cluster deletion , the local pvs ("local-pv-*") went into "Failed" status.

]# oc get pvc -A
No resources found

# oc get pv -A
NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                      STORAGECLASS   REASON   AGE
local-pv-159dec11   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-11pwnkr   localdisk               47h
local-pv-3d3997df   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-976qjn    localdisk               47h
local-pv-449b7f53   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-1jbfnv    localdisk               47h
local-pv-47f41b17   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-2bqh4s    localdisk               47h
local-pv-6ebd0167   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-48jthk    localdisk               47h
local-pv-738e6db8   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-0fjlp2    localdisk               47h
local-pv-8c29b3ff   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-102b2cg   localdisk               47h
local-pv-b1b024bb   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-7qjrcv    localdisk               47h
local-pv-bc9e1249   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-89m79h    localdisk               47h
local-pv-bdf88db2   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-6nqw6s    localdisk               47h
local-pv-e2ea2c34   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-3h5dqk    localdisk               47h
local-pv-ed12a655   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-5tcwqq    localdisk               47h



Version of all relevant components (if applicable):
OCP - 4.8.0-0.nightly-s390x-2021-05-27-091328
OCS - 4.8.0-404.ci 
LSO - 4.8.0-202105191051.p0.assembly.stream 


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Workaround can be applied

Is there any workaround available to the best of your knowledge?
Yes

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
3

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1.Install OCS
2.Delete the Storage Cluster object and wait for the removal of the associated resources.

$ oc delete -n openshift-storage storagecluster --all --wait=true

3. Check the status of the localpvs


Actual results:
Pvc deletion failed to release claim and left the PVs in "Failed" status

#oc get pv -A
NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                      STORAGECLASS   REASON   AGE
local-pv-159dec11   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-11pwnkr   localdisk               47h
local-pv-3d3997df   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-976qjn    localdisk               47h
local-pv-449b7f53   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-1jbfnv    localdisk               47h
local-pv-47f41b17   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-2bqh4s    localdisk               47h
local-pv-6ebd0167   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-48jthk    localdisk               47h
local-pv-738e6db8   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-0fjlp2    localdisk               47h
local-pv-8c29b3ff   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-102b2cg   localdisk               47h
local-pv-b1b024bb   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-7qjrcv    localdisk               47h
local-pv-bc9e1249   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-89m79h    localdisk               47h
local-pv-bdf88db2   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-6nqw6s    localdisk               47h
local-pv-e2ea2c34   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-3h5dqk    localdisk               47h
local-pv-ed12a655   1Ti        RWO            Delete           Failed   openshift-storage/ocs-deviceset-localdisk-0-data-5tcwqq    localdisk               47h

Expected results:

local-pvs should be in "Available" status

Additional info:

Comment 2 Sravika 2021-06-02 14:45:21 UTC
Created attachment 1788714 [details]
Rook Operator log

Comment 3 Jan Safranek 2021-06-08 08:38:53 UTC
In the PVs I can see:

> status:
>   message: 'Error getting deleter volume plugin for volume "local-pv-159dec11": no deletable volume plugin matched'

We're fixing it in bug #1952820 - please retest with an updated version and feel free to reopen if it does not work.

*** This bug has been marked as a duplicate of bug 1952820 ***

Comment 4 Sravika 2021-07-27 08:01:31 UTC
@jsafrane: With the latest fix, I can see that the pvc deletion released the claim successfully. 
However after the claim is Released the PVs were deleted and freshly created and are in status "Available".
Is this expected behaviour?


Note You need to log in before you can comment on or make changes to this bug.