Description of problem: RHV to CNV import using Ceph-rbd/Filesystem RWO fails due not enough space on created device $ oc describe pvc v2v-fedora32-8737b4f7-2b6b-4801-abc9-e307f838b337 Name: v2v-fedora32-8737b4f7-2b6b-4801-abc9-e307f838b337 Namespace: default StorageClass: ocs-storagecluster-ceph-rbd Status: Bound Volume: pvc-91f77420-0555-4f21-96d7-f0e0662fa316 Labels: app=containerized-data-importer Annotations: cdi.kubevirt.io/storage.condition.running: true cdi.kubevirt.io/storage.condition.running.message: cdi.kubevirt.io/storage.condition.running.reason: Pod is running cdi.kubevirt.io/storage.import.certConfigMap: vmimport.v2v.kubevirt.iolw2vx cdi.kubevirt.io/storage.import.diskId: 8737b4f7-2b6b-4801-abc9-e307f838b337 cdi.kubevirt.io/storage.import.endpoint: https://rhev-blue-01.rdu2.scalelab.redhat.com/ovirt-engine/api cdi.kubevirt.io/storage.import.importPodName: importer-v2v-fedora32-8737b4f7-2b6b-4801-abc9-e307f838b337 cdi.kubevirt.io/storage.import.secretName: vmimport.v2v.kubevirt.iobjn29 cdi.kubevirt.io/storage.import.source: imageio cdi.kubevirt.io/storage.pod.phase: Running cdi.kubevirt.io/storage.pod.restarts: 4 pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.beta.kubernetes.io/storage-provisioner: openshift-storage.rbd.csi.ceph.com Finalizers: [kubernetes.io/pvc-protection] Capacity: 4Gi Access Modes: RWO VolumeMode: Filesystem Mounted By: importer-v2v-fedora32-8737b4f7-2b6b-4801-abc9-e307f838b337 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ExternalProvisioning 3m18s (x2 over 3m18s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "openshift-storage.rbd.csi.ceph.com" or manually created by system administrator Normal Provisioning 3m18s openshift-storage.rbd.csi.ceph.com_csi-rbdplugin-provisioner-bc6b69c69-d2w7j_2bc852f9-41b0-40de-9719-786c9127437c External provisioner is provisioning volume for claim "default/v2v-fedora32-8737b4f7-2b6b-4801-abc9-e307f838b337" Normal ProvisioningSucceeded 3m18s openshift-storage.rbd.csi.ceph.com_csi-rbdplugin-provisioner-bc6b69c69-d2w7j_2bc852f9-41b0-40de-9719-786c9127437c Successfully provisioned volume pvc-91f77420-0555-4f21-96d7-f0e0662fa316 Warning ErrImportFailed 107s (x4 over 108s) import-controller Unable to process data: write /data/disk.img: no space left on device Warning ErrImportFailed 21s (x14 over 92s) import-controller Unable to connect to imageio data source: Fault reason is "Operation Failed". Fault detail is "[Cannot transfer Virtual Disk: The following disks are locked: GlanceDisk-f6c31e5. Please try again in a few minutes.]". HTTP response code is "409". HTTP response message is "409 Conflict". Version-Release number of selected component (if applicable): CNV 2.5.0 OCP 4.6.0-fc.8 How reproducible: 100% Actual results: By Alex Wels: VM importer is doing a perfectly normal assumption to take the disk size from the source as the disk size to use to import. CDI is requesting a PVC of that size. Ceph is creating a block volume (because it can only create block). But we requested a file system, so after the volume is created, it adds a file system, that is where the problem happens - The file system has overhead (for things like inodes, and other bookkeeping). So the actual available space on the file system that is returned is smaller than the requested size Expected results: To request PVC with larger size than the source VM disk while using Ceph-RBD/Filesystem for import Additional info: For Ceph-RBD Filesystem only RWO AccessMode should work
@Piotr, assigning to you as it's on the VMIO side. When the controller creates the DataVolume, it should request a size slightly larger than the source disk. And I think it also concerns all block backend with a filesystem on top.
@Fabien, This is an issue in CDI we have no ability to fix this in VMIO. Changing the component and assignee.
In CNV-2.6 we will provide you with an API in CDIConfig where you can look up the recommended FS overhead value. You should grab this value and inflate the size of the DV/PVC yourself.