+++ This bug was initially created as a clone of Bug #1459461 +++ Description of problem: Whenever oc delete pvc command is run, pvc gets deleted successfully, but the deletion of pv is failing Version-Release number of selected component (if applicable): cns-3.6 How reproducible: Everytime Steps to Reproduce: 1. Create a SC 2. Create a PV, PVC 3. Delete a PVC Actual results: The volumes corresponding to PVC should be deleted Expected results: PVC is getting deleted but the PV goes into failed state Additional info: [root ~]# oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE storage-claim1 Bound pvc-c9cd9320-4b38-11e7-ba89-005056848cc9 1Gi RWO fast 1h storage-claim2 Bound pvc-d017751e-4b38-11e7-ba89-005056848cc9 1Gi RWO fast 1h storage-claim3 Bound pvc-d65c5b26-4b38-11e7-ba89-005056848cc9 2Gi RWO fast 1h [root ~]# oc get pv NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-c9cd9320-4b38-11e7-ba89-005056848cc9 1Gi RWO Delete Bound storage-project/storage-claim1 fast 1h pvc-d017751e-4b38-11e7-ba89-005056848cc9 1Gi RWO Delete Bound storage-project/storage-claim2 fast 1h pvc-d65c5b26-4b38-11e7-ba89-005056848cc9 2Gi RWO Delete Bound storage-project/storage-claim3 fast 1h [root ~]# oc delete pvc storage-claim3 persistentvolumeclaim "storage-claim3" deleted [root ~]# oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE storage-claim1 Bound pvc-c9cd9320-4b38-11e7-ba89-005056848cc9 1Gi RWO fast 1h storage-claim2 Bound pvc-d017751e-4b38-11e7-ba89-005056848cc9 1Gi RWO fast 1h [root ~]# oc get pv NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-c9cd9320-4b38-11e7-ba89-005056848cc9 1Gi RWO Delete Bound storage-project/storage-claim1 fast 1h pvc-d017751e-4b38-11e7-ba89-005056848cc9 1Gi RWO Delete Bound storage-project/storage-claim2 fast 1h pvc-d65c5b26-4b38-11e7-ba89-005056848cc9 2Gi RWO Delete Failed storage-project/storage-claim3 fast 1h [root ~]# oc describe pv pvc-d65c5b26-4b38-11e7-ba89-005056848cc9 Name: pvc-d65c5b26-4b38-11e7-ba89-005056848cc9 Labels: <none> Annotations: pv.beta.kubernetes.io/gid=2002 pv.kubernetes.io/bound-by-controller=yes pv.kubernetes.io/provisioned-by=kubernetes.io/glusterfs StorageClass: fast Status: Failed Claim: storage-project/storage-claim3 Reclaim Policy: Delete Access Modes: RWO Capacity: 2Gi Message: Volume has no class annotation Source: Type: Glusterfs (a Glusterfs mount on the host that shares a pod's lifetime) EndpointsName: glusterfs-dynamic-storage-claim3 Path: vol_6c6b19af3c63069854d33f57fce9c032 ReadOnly: false Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 33s 33s 1 persistent-volume-controller Warning VolumeFailedDelete Volume has no class annotation --- Additional comment from Red Hat Bugzilla Rules Engine on 2017-06-07 04:41:38 EDT --- This bug is automatically being proposed for the current release of Container-Native Storage under active development, by setting the release flag 'cns‑3.6.0' to '?'. If this bug should be proposed for a different release, please manually change the proposed release flag. --- Additional comment from Atin Mukherjee on 2017-06-07 05:24:16 EDT --- If this is done with brick multiplexing being enabled then I guess this is a duplicate of BZ 1444749 which now has been fixed in RHGS through BZ 1457219. CNS team needs to rebuild the image with glusterfs-3.8.4-27. --- Additional comment from Mohamed Ashiq on 2017-06-08 06:52:41 EDT --- (In reply to Atin Mukherjee from comment #2) > If this is done with brick multiplexing being enabled then I guess this is a > duplicate of BZ 1444749 which now has been fixed in RHGS through BZ 1457219. > CNS team needs to rebuild the image with glusterfs-3.8.4-27. This is not related to brick multiplexing. This because of annotations missing in the dynamically created pv's. Refer: https://github.com/kubernetes/kubernetes/issues/43929 workaround: oc edit pv <pv-name> Add the below line in annotation section: volume.beta.kubernetes.io/storage-class: "<strageclass-name>" This will be fixed in openshift. I will keep track of the fix. --- Additional comment from Mohamed Ashiq on 2017-06-08 07:40:23 EDT --- Fix is in kubernetes upstream but not yet in openshift upstream. https://github.com/kubernetes/kubernetes/pull/44035/files Issue is because annotations missing in the dynamically created pv's. Refer: https://github.com/kubernetes/kubernetes/issues/43929 workaround: oc edit pv <pv-name> Add the below line in annotation section: volume.beta.kubernetes.io/storage-class: "<strageclass-name>" This will be fixed in openshift. I will keep track of the fix. --- Additional comment from Humble Chirammal on 2017-06-15 01:41:59 EDT --- Tejas, Can you please check whether this issue is fixed in latest OCP builds ? --- Additional comment from Humble Chirammal on 2017-06-15 01:58:34 EDT --- (In reply to Humble Chirammal from comment #5) > Tejas, Can you please check whether this issue is fixed in latest OCP builds > ? Also please let me know the OCP build version used in your setup. --- Additional comment from Jan Safranek on 2017-06-15 06:11:23 EDT --- This indeed looks like a bug in OpenShift, current 3.6 does not have https://github.com/kubernetes/kubernetes/pull/43982 --- Additional comment from Humble Chirammal on 2017-06-15 06:16:13 EDT --- Thanks Jan for update and filing PR # https://github.com/openshift/origin/pull/14667 --- Additional comment from Jan Safranek on 2017-06-15 06:19:09 EDT --- Origin PR: https://github.com/openshift/origin/pull/14667 --- Additional comment from Humble Chirammal on 2017-06-16 01:40:22 EDT --- Above PR is merged, so this issue should be fixed from next OCP 3.6 builds. --- Additional comment from on 2017-06-16 12:13:10 EDT ---
This bug has been fixed in latest OCP version, iic, krk has verified it.
(In reply to Humble Chirammal from comment #4) > This bug has been fixed in latest OCP version, iic, krk has verified it. https://bugzilla.redhat.com/show_bug.cgi?id=1444749#c38 . I am closing this bug for now and please feel free to reopen if accordingly. *** This bug has been marked as a duplicate of bug 1444749 ***
*** This bug has been marked as a duplicate of bug 1462138 ***