Description of problem: LocalVolume PV could not be deleted. Version-Release number of selected component (if applicable): local-storage-operator.4.8.0-202104212208.p0 4.8.0-0.nightly-2021-04-22-061234 How reproducible: Always Steps to Reproduce: 1.Deploy LocalStorageOperator 2.Create CR Localvolume apiVersion: local.storage.openshift.io/v1 kind: LocalVolume metadata: name: example7 namespace: openshift-local-storage spec: logLevel: Normal managementState: Managed storageClassDevices: - devicePaths: - /dev/disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0e630a6f37ce08555 - /dev/disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0db336e047afe9b1d storageClassName: lv-sc7 3.Check storageclass oc get sc lv-sc7 -o json | jq .reclaimPolicy "Delete" 4.Create pvc/pod local-pv-85bb6989 3Gi RWO Delete Bound openshift-local-storage/pvc2 lv-sc7 58m 5.Delete pod/pvc local-pv-85bb6989 3Gi RWO Delete Failed openshift-local-storage/pvc2 lv-sc7 70m 6.oc describe pv/local-pv-85bb6989 Name: local-pv-85bb6989 Labels: kubernetes.io/hostname=ip-10-0-158-253 storage.openshift.com/local-volume-owner-name=example7 storage.openshift.com/local-volume-owner-namespace=openshift-local-storage storage.openshift.com/owner-kind=LocalVolume storage.openshift.com/owner-name=example7 storage.openshift.com/owner-namespace=openshift-local-storage Annotations: pv.kubernetes.io/bound-by-controller: yes storage.openshift.com/device-id: nvme-Amazon_Elastic_Block_Store_vol0db336e047afe9b1d storage.openshift.com/device-name: nvme3n1 Finalizers: [kubernetes.io/pv-protection] StorageClass: lv-sc7 Status: Failed Claim: openshift-local-storage/pvc2 Reclaim Policy: Delete Access Modes: RWO VolumeMode: Filesystem Capacity: 3Gi Node Affinity: Required Terms: Term 0: kubernetes.io/hostname in [ip-10-0-158-253] Message: Error getting deleter volume plugin for volume "local-pv-85bb6989": no deletable volume plugin matched Source: Type: LocalVolume (a persistent volume backed by local storage on a node) Path: /mnt/local-storage/lv-sc7/nvme-Amazon_Elastic_Block_Store_vol0db336e047afe9b1d Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning VolumeFailedDelete 12m persistentvolume-controller Error getting deleter volume plugin for volume "local-pv-85bb6989": no deletable volume plugin matched Actual results: PV is in Failed status. Expected results: PV should be deleted. Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info:
There is no provisioned-by annotation, it must be set to get deletion to work. Did we remove it recently?
oc describe pv Name: local-pv-179b1bd3 Labels: kubernetes.io/hostname=ip-10-0-169-208 storage.openshift.com/local-volume-owner-name=example storage.openshift.com/local-volume-owner-namespace=openshift-local-storage storage.openshift.com/owner-kind=LocalVolume storage.openshift.com/owner-name=example storage.openshift.com/owner-namespace=openshift-local-storage Annotations: pv.kubernetes.io/bound-by-controller: yes pv.kubernetes.io/provisioned-by: local-volume-provisioner-ip-10-0-169-208.us-east-2.compute.internal-8670c4d1-d8b2-43ac-8fa2-dd66021d88ec storage.openshift.com/device-id: nvme-Amazon_Elastic_Block_Store_vol0f3dd3dc00f752b37 storage.openshift.com/device-name: nvme1n1 Finalizers: [kubernetes.io/pv-protection] StorageClass: lv-sc Status: Released Claim: openshift-local-storage/pvc4 Reclaim Policy: Delete Access Modes: RWO VolumeMode: Filesystem Capacity: 2Gi Node Affinity: Required Terms: Term 0: kubernetes.io/hostname in [ip-10-0-169-208] Message: Source: Type: LocalVolume (a persistent volume backed by local storage on a node) Path: /mnt/local-storage/lv-sc/nvme-Amazon_Elastic_Block_Store_vol0f3dd3dc00f752b37 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal VolumeDelete 15s deleter Starting cleanup of Block PV "local-pv-179b1bd3", this may take a while oc get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE local-pv-179b1bd3 2Gi RWO Delete Available lv-sc 5s local-storage-operator.4.8.0-202105242248.p0.assembly.stream
*** Bug 1967175 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438