Bug 1829438

+++ This bug was initially created as a clone of Bug #1829376 +++

Description of problem:
We're attempting to spin up a VM backed by a DataVolume. If that DataVolume is set with volumeMode: block and the source is blank, the PVC and DV get created, but no VM pod ever spins up.

Version-Release number of selected component (if applicable):
hyperconverged-cluster-operator:v2.2.0-12
virt-cdi-operator:v2.2.0-5


How reproducible:
Always

Steps to Reproduce:
1. Create a VM template with a DV, with volumeMode: block and source: blank: {}
2. oc create -f <vm template>


Actual results:
PVC is created
DV is created, but with status:
  status:
    phase: PVCBound
rather than:
  status:
    phase: Succeeded
    progress: 100.0%

VM is created and says it is running, but no VMI/pod exist:

# oc get vm/dc-vm-2
NAME      AGE     RUNNING   VOLUME
dc-vm-2   7m12s   true      




Expected results:
DV 'succeeds' and the VM starts

Additional info:
debugging info requested in email:
# cat dv-test.yml
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: dv-block-blank
spec:
  pvc:
    accessModes:
    - ReadWriteMany
    dataSource: null
    resources:
      requests:
        storage: 10Gi
    storageClassName: ocs-storagecluster-ceph-rbd
    volumeMode: Block
  source:
    blank: {}
# oc create -f dv-test.yml
datavolume.cdi.kubevirt.io/dv-block-blank created
# oc get dv/dv-block-blank -o yaml
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  creationTimestamp: "2020-04-29T00:08:02Z"
  generation: 3
  name: dv-block-blank
  namespace: ocs-cnv
  resourceVersion: "26471071"
  selfLink: /apis/cdi.kubevirt.io/v1alpha1/namespaces/ocs-cnv/datavolumes/dv-block-blank
  uid: 009892d3-1cdd-4f14-a7c0-a13f0abd9a34
spec:
  pvc:
    accessModes:
    - ReadWriteMany
    dataSource: null
    resources:
      requests:
        storage: 10Gi
    storageClassName: ocs-storagecluster-ceph-rbd
    volumeMode: Block
  source:
    blank: {}
status:
  phase: PVCBound
# oc get pvc/dv-block-blank -o yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    cdi.kubevirt.io/storage.contentType: kubevirt
    cdi.kubevirt.io/storage.import.source: none
    cdi.kubevirt.io/storage.pod.phase: Succeeded
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
    volume.beta.kubernetes.io/storage-provisioner: openshift-storage.rbd.csi.ceph.com
  creationTimestamp: "2020-04-29T00:08:02Z"
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app: containerized-data-importer
    cdi-controller: dv-block-blank
  name: dv-block-blank
  namespace: ocs-cnv
  ownerReferences:
  - apiVersion: cdi.kubevirt.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: DataVolume
    name: dv-block-blank
    uid: 009892d3-1cdd-4f14-a7c0-a13f0abd9a34
  resourceVersion: "26471082"
  selfLink: /api/v1/namespaces/ocs-cnv/persistentvolumeclaims/dv-block-blank
  uid: 096fff16-acf3-424e-9012-4bcc89a754c9
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: ocs-storagecluster-ceph-rbd
  volumeMode: Block
  volumeName: pvc-096fff16-acf3-424e-9012-4bcc89a754c9
status:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  phase: Bound
# oc get dv/dv-block-blank -o yaml
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  creationTimestamp: "2020-04-29T00:22:38Z"
  generation: 3
  name: dv-block-blank
  namespace: ocs-cnv
  resourceVersion: "26479912"
  selfLink: /apis/cdi.kubevirt.io/v1alpha1/namespaces/ocs-cnv/datavolumes/dv-block-blank
  uid: 8305760e-ef32-4c7a-9b3c-82dc40990949
spec:
  pvc:
    accessModes:
    - ReadWriteOnce
    dataSource: null
    resources:
      requests:
        storage: 10Gi
    storageClassName: ocs-storagecluster-ceph-rbd
    volumeMode: Block
  source:
    blank: {}
status:
  phase: Succeeded
  progress: 100.0%

--- Additional comment from Alexander Wels on 2020-04-29 14:06:52 UTC ---

This is a bug in CDI, it is not always updating the phase of the DV to be successful, which is causing KubeVirt to not spin up a VMI. The VM will have running set to true, which normally would create the VMI, but due to the DV not being in succeeded causes the VMI to not get created.

--- Additional comment from Adam Litke on 2020-04-29 14:21:15 UTC ---

Workaround: Creating a blank disk on a block volume is a no-op for CDI since the empty block device can be used directly by qemu.  Therefore, as a workaround for this problem you can directly use a PVC as the disk in your VM spec.  For example:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-blank-disk
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: ocs-storagecluster-ceph-rbd
  volumeMode: Block

and use it in your VM like so:

...
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: virtio
            name: blankdisk
...
      volumes:
      - name: blankdisk
        persistentVolumeClaim:
          claimName: pvc-blank-disk
...