Bug 1588453

This is continuation of bug #1569404.

Cinder driver we use for testing is broken:

https://github.com/kubernetes/cloud-provider-openstack/issues/150

It trusts Cinder that it returns correct device name for volumes. This is not true in OpenStack + KVM combo - Cinder sometimes returns wrong device name there.

In-tree Cinder driver has a code that finds the right volume despite Cinder returning a wrong one.

Current CSI driver does not have such code. In SetUp call, it waits until the (wrong) device appears. It never appears and kubelet times out and pod is ContainerCreating forever.

So far so good (at least on Kubernetes side). Kubernetes should recover from that. The bug is that it does *not* recover when user deletes the pod.

-> Kubelet calls CSI's UnmountDevice()
  -> UnmountDevice() calls PVs.Get() to get coordinates of the volume (i.e. 
driver name and volume handle)
    -> The pod is already deleted, so Kubelet can't read the PV
      -> error

csi_attacher.go:471] kubernetes.io/csi: attacher.UnmountDevice failed to get driver and volume name from device mount path: persistentvolumes "kubernetes-dynamic-pv-b2f4b7ad6a4511e8" is forbidden:  User "system:node:qe-juzhao-310-qeos-2-nrr-1" cannot get persistentvolumes at the cluster scope: no path found to object

The volume is never removed from volumesInUse. The volume is forcefully detached by A/D controller when it gives up waiting for kubelet (5 minutes?).

Version-Release number of selected component (if applicable):
openshift v3.10.0-0.63.0