Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1461688

Summary:	[rbd] Failed to reclaim RBD PV: "volume has no class annotation"
Product:	OpenShift Container Platform	Reporter:	Jianwei Hou <jhou>
Component:	Storage	Assignee:	Hemant Kumar <hekumar>
Status:	CLOSED DUPLICATE	QA Contact:	Jianwei Hou <jhou>
Severity:	high	Docs Contact:
Priority:	high
Version:	3.6.0	CC:	aos-bugs, hchen, hekumar
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-06-16 16:13:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jianwei Hou 2017-06-15 07:58:07 UTC

Description of problem:
The RBD PV created by dynamic provisioner can not be automatically deleted after deleting PVC. Error "Failed to create deleter for volume "pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a": Volume has no class annotation"

Version-Release number of selected component (if applicable):
openshift v3.6.106
kubernetes v1.6.1+5115d708d7
etcd 3.2.0

How reproducible:
Always

Steps to Reproduce:
1. Create StorageClass for Ceph RBD dynamic provisioner.
2. Create PVC
3. Delete the PVC

Actual results:
PV can not be automatically deleted.

Expected results:
PV should be deleted.

Master Log:
Jun 15 03:55:43 ip-172-18-5-248 atomic-openshift-master: I0615 03:55:43.681862   32272 pv_controller.go:956] reclaimVolume[pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a]: policy is Delete
Jun 15 03:55:43 ip-172-18-5-248 atomic-openshift-master: I0615 03:55:43.681868   32272 pv_controller.go:1408] scheduleOperation[delete-pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a[1ce5c247-519f-11e7-b651-0ed38aa7e56a]]
Jun 15 03:55:43 ip-172-18-5-248 atomic-openshift-master: I0615 03:55:43.681876   32272 pv_controller.go:1421] operation "delete-pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a[1ce5c247-519f-11e7-b651-0ed38aa7e56a]" postponed due to exponential backoff


PV Dump:
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/bound-by-controller: "yes"
    pv.kubernetes.io/provisioned-by: kubernetes.io/rbd
  creationTimestamp: 2017-06-15T07:49:08Z
  name: pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a
  resourceVersion: "7857"
  selfLink: /api/v1/persistentvolumes/pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a
  uid: 1ce5c247-519f-11e7-b651-0ed38aa7e56a
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 1Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: rbdc1
    namespace: jhou
    resourceVersion: "7844"
    uid: 1cb91163-519f-11e7-b651-0ed38aa7e56a
  persistentVolumeReclaimPolicy: Delete
  rbd:
    image: kubernetes-dynamic-pvc-1cc17086-519f-11e7-b651-0ed38aa7e56a
    keyring: /etc/ceph/keyring
    monitors:
    - ip-172-18-14-142.ec2.internal:6789
    pool: rbd
    secretRef:
      name: cephrbd-user-secret
    user: admin
  storageClassName: cephrbdprovisioner
status:
  message: 'Failed to create deleter for volume "pvc-1cb91163-519f-11e7-b651-0ed38aa7e56a":
    Volume has no class annotation'
  phase: Failed

PVC Dump:
{
   "kind": "PersistentVolumeClaim",
   "apiVersion": "v1",
   "metadata": {
     "name": "rbdc1",
     "annotations": {
     "volume.beta.kubernetes.io/storage-class": "cephrbdprovisioner"
     }
   },
   "spec": {
     "accessModes": [
       "ReadWriteOnce"
     ],
     "resources": {
       "requests": {
         "storage": "1Gi"
       }
     }
   }
}


StorageClass Dump (if StorageClass used by PV/PVC):
apiVersion: storage.k8s.io/v1beta1
kind: StorageClass
metadata:
  name: cephrbdprovisioner
provisioner: kubernetes.io/rbd
parameters:
  monitors: "ip-172-18-14-142.ec2.internal:6789"
  adminId: "admin"
  adminSecretName: "cephrbd-secret"
  adminSecretNamespace: "default"
  pool: "rbd"
  userId: "admin"
  userSecretName: "cephrbd-user-secret"

Additional info: