Bug 1952820
| Summary: | [LSO] Delete localvolume pv is failed | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Chao Yang <chaoyang> |
| Component: | Storage | Assignee: | Rohan CJ <rojoseph> |
| Storage sub component: | Local Storage Operator | QA Contact: | Chao Yang <chaoyang> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | high | ||
| Priority: | unspecified | CC: | aos-bugs, hekumar, jsafrane, rkant, sbalusu, ssonigra |
| Version: | 4.8 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-07-27 23:03:27 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1937145, 1971231 | ||
There is no provisioned-by annotation, it must be set to get deletion to work. Did we remove it recently? oc describe pv
Name: local-pv-179b1bd3
Labels: kubernetes.io/hostname=ip-10-0-169-208
storage.openshift.com/local-volume-owner-name=example
storage.openshift.com/local-volume-owner-namespace=openshift-local-storage
storage.openshift.com/owner-kind=LocalVolume
storage.openshift.com/owner-name=example
storage.openshift.com/owner-namespace=openshift-local-storage
Annotations: pv.kubernetes.io/bound-by-controller: yes
pv.kubernetes.io/provisioned-by: local-volume-provisioner-ip-10-0-169-208.us-east-2.compute.internal-8670c4d1-d8b2-43ac-8fa2-dd66021d88ec
storage.openshift.com/device-id: nvme-Amazon_Elastic_Block_Store_vol0f3dd3dc00f752b37
storage.openshift.com/device-name: nvme1n1
Finalizers: [kubernetes.io/pv-protection]
StorageClass: lv-sc
Status: Released
Claim: openshift-local-storage/pvc4
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 2Gi
Node Affinity:
Required Terms:
Term 0: kubernetes.io/hostname in [ip-10-0-169-208]
Message:
Source:
Type: LocalVolume (a persistent volume backed by local storage on a node)
Path: /mnt/local-storage/lv-sc/nvme-Amazon_Elastic_Block_Store_vol0f3dd3dc00f752b37
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal VolumeDelete 15s deleter Starting cleanup of Block PV "local-pv-179b1bd3", this may take a while
oc get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
local-pv-179b1bd3 2Gi RWO Delete Available lv-sc 5s
local-storage-operator.4.8.0-202105242248.p0.assembly.stream
*** Bug 1967175 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |
Description of problem: LocalVolume PV could not be deleted. Version-Release number of selected component (if applicable): local-storage-operator.4.8.0-202104212208.p0 4.8.0-0.nightly-2021-04-22-061234 How reproducible: Always Steps to Reproduce: 1.Deploy LocalStorageOperator 2.Create CR Localvolume apiVersion: local.storage.openshift.io/v1 kind: LocalVolume metadata: name: example7 namespace: openshift-local-storage spec: logLevel: Normal managementState: Managed storageClassDevices: - devicePaths: - /dev/disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0e630a6f37ce08555 - /dev/disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0db336e047afe9b1d storageClassName: lv-sc7 3.Check storageclass oc get sc lv-sc7 -o json | jq .reclaimPolicy "Delete" 4.Create pvc/pod local-pv-85bb6989 3Gi RWO Delete Bound openshift-local-storage/pvc2 lv-sc7 58m 5.Delete pod/pvc local-pv-85bb6989 3Gi RWO Delete Failed openshift-local-storage/pvc2 lv-sc7 70m 6.oc describe pv/local-pv-85bb6989 Name: local-pv-85bb6989 Labels: kubernetes.io/hostname=ip-10-0-158-253 storage.openshift.com/local-volume-owner-name=example7 storage.openshift.com/local-volume-owner-namespace=openshift-local-storage storage.openshift.com/owner-kind=LocalVolume storage.openshift.com/owner-name=example7 storage.openshift.com/owner-namespace=openshift-local-storage Annotations: pv.kubernetes.io/bound-by-controller: yes storage.openshift.com/device-id: nvme-Amazon_Elastic_Block_Store_vol0db336e047afe9b1d storage.openshift.com/device-name: nvme3n1 Finalizers: [kubernetes.io/pv-protection] StorageClass: lv-sc7 Status: Failed Claim: openshift-local-storage/pvc2 Reclaim Policy: Delete Access Modes: RWO VolumeMode: Filesystem Capacity: 3Gi Node Affinity: Required Terms: Term 0: kubernetes.io/hostname in [ip-10-0-158-253] Message: Error getting deleter volume plugin for volume "local-pv-85bb6989": no deletable volume plugin matched Source: Type: LocalVolume (a persistent volume backed by local storage on a node) Path: /mnt/local-storage/lv-sc7/nvme-Amazon_Elastic_Block_Store_vol0db336e047afe9b1d Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning VolumeFailedDelete 12m persistentvolume-controller Error getting deleter volume plugin for volume "local-pv-85bb6989": no deletable volume plugin matched Actual results: PV is in Failed status. Expected results: PV should be deleted. Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info: