Created attachment 1768054 [details] Pod's log with errors Created attachment 1768054 [details] Pod's log with errors Description of problem: Migration from RHV fails if using NFS as a target storage Version-Release number of selected component (if applicable): CNV 2.6.1 How reproducible: 100% Steps to Reproduce: 1. Start migration wizard as usual 2. When defining target storage - use NFS with filesystem 3. Start migration and take a look on import log Actual results: Migration will fail on resizing target image Expected results: Migration succeeded Additional info: I0331 08:51:16.058986 1 data-processor.go:239] New phase: Resize W0331 08:51:16.423118 1 data-processor.go:331] Available space less than requested size, resizing image to available space 4281974784. I0331 08:51:16.423155 1 data-processor.go:337] Expanding image size to: 4281974784 E0331 08:51:16.438761 1 prlimit.go:174] qemu-img failed output is: E0331 08:51:16.438789 1 prlimit.go:175] E0331 08:51:16.438796 1 prlimit.go:176] qemu-img: Use the --shrink option to perform a shrink operation. qemu-img: warning: Shrinking an image will delete all data beyond the shrunken image's end. Before performing such an operation, make sure there is no important data there. E0331 08:51:16.438856 1 data-processor.go:236] exit status 1 qemu-img execution failed
Same behavior with cephs/Filesystem (while cephs/block has no issues) Maybe this is the issue.
I backported the PR that introduced this breakage to CDI: https://github.com/kubevirt/containerized-data-importer/pull/1612 I am happy to rip it out of 2.6.1 if it comes to that, there is no strong reason to backport it. But that will push out the problem to 4.8, not make it go away.
Can I get a copy of the disk image that is written to the PVC? The PVC is already converted, but not yet resized. That way I can use it to track down the problem. Also I would like a copy of the DV that creates the PVC.
Created attachment 1768168 [details] dv_yaml
This is the workaround: oc patch cdi cdi-kubevirt-hyperconverged --type merge -p '{"spec": {"config": {"filesystemOverhead": {"global": "0"}}}}' This will unset the value: oc patch cdi cdi-kubevirt-hyperconverged --type merge -p '{"spec": {"config": {"filesystemOverhead": {}}}}' Does this work around work?
Thanks Maya, The work around works fine. I used a source VM fedora32 4G disk and run those steps: 1. Without running the workaround applied yet, VM import from RHV to destination storage NFS/Filesystem-> Failed at the resize phase, as expected. 2. Apply the workaround 3. VM import from RHV to destination storage NFS/Filesystem ->PASS, VM started OK. 4. VM import from RHV to destination storage Ceph-RBD/Block ->PASS. 5. Run the command to unset the value - If here the intention was to return the behavior to how it was before applying the work around then this didn't work. VM import from RHV to NFS still PASSES. (Looks like the resize phase is still not running, but I didn't catch the exact log last message on this). Maya, Can you advise on this?
Add: oc patch cdi cdi-kubevirt-hyperconverged --type merge -p '{"spec": {"config": {"filesystemOverhead": {"global": "0"}}}}' Remove: oc patch cdi cdi-kubevirt-hyperconverged --type=json -p '[{ "op": "remove", "path": "/spec/config/filesystemOverhead" }]'
Fabien, Could we use this bug to track delivery of the size calculation in vm import controller for 2.6.2? If there is a PR, could you attach it here?
Good idea. I've linked the PR.
@Fabien, Can this bug move to ON_QA already?
Yes, the latest build should have the fix.
Verified on OCP-4.7.7/CNV: iib-67945 hco-v2.6.2-12 importer.log: I0420 17:36:04.087307 1 importer.go:52] Starting importer I0420 17:36:04.088055 1 importer.go:134] begin import process I0420 17:36:04.917211 1 http-datasource.go:214] Attempting to get certs from /certs/ca.pem I0420 17:36:04.943675 1 data-processor.go:357] Calculating available size I0420 17:36:04.944865 1 data-processor.go:369] Checking out file system volume size. I0420 17:36:04.945235 1 data-processor.go:377] Request image size not empty. I0420 17:36:04.945248 1 data-processor.go:382] Target size 11362347520. I0420 17:36:04.946643 1 data-processor.go:239] New phase: TransferDataFile I0420 17:36:04.949269 1 util.go:161] Writing data... I0420 17:36:05.948957 1 prometheus.go:69] 2.97 I0420 17:36:06.949403 1 prometheus.go:69] 5.95 I0420 17:36:07.949536 1 prometheus.go:69] 9.08 I0420 17:36:08.950759 1 prometheus.go:69] 12.35 I0420 17:36:09.951650 1 prometheus.go:69] 15.78 I0420 17:36:10.951794 1 prometheus.go:69] 19.15 I0420 17:36:11.951951 1 prometheus.go:69] 22.59 I0420 17:36:12.956927 1 prometheus.go:69] 26.09 I0420 17:36:13.957178 1 prometheus.go:69] 29.63 I0420 17:36:14.957385 1 prometheus.go:69] 33.12 I0420 17:36:15.957595 1 prometheus.go:69] 36.72 I0420 17:36:16.957777 1 prometheus.go:69] 40.31 I0420 17:36:17.958021 1 prometheus.go:69] 43.75 I0420 17:36:18.958110 1 prometheus.go:69] 47.27 I0420 17:36:19.958278 1 prometheus.go:69] 50.81 I0420 17:36:20.962524 1 prometheus.go:69] 54.19 I0420 17:36:21.962633 1 prometheus.go:69] 57.74 I0420 17:36:22.962781 1 prometheus.go:69] 61.21 I0420 17:36:23.964158 1 prometheus.go:69] 64.61 I0420 17:36:24.964298 1 prometheus.go:69] 68.05 I0420 17:36:25.964433 1 prometheus.go:69] 71.52 I0420 17:36:26.964517 1 prometheus.go:69] 74.84 I0420 17:36:27.964631 1 prometheus.go:69] 78.25 I0420 17:36:28.964746 1 prometheus.go:69] 81.61 I0420 17:36:29.969447 1 prometheus.go:69] 85.00 I0420 17:36:30.969784 1 prometheus.go:69] 88.10 I0420 17:36:31.969891 1 prometheus.go:69] 91.29 I0420 17:36:32.970111 1 prometheus.go:69] 94.22 I0420 17:36:33.970210 1 prometheus.go:69] 97.52 I0420 17:36:34.970295 1 prometheus.go:69] 100.00 I0420 17:36:37.255459 1 data-processor.go:239] New phase: Resize W0420 17:36:38.085904 1 data-processor.go:343] Available space less than requested size, resizing image to available space 10737418240. I0420 17:36:38.085926 1 data-processor.go:346] No need to resize image. Requested size: 11362347520, Image size: 10737418240. I0420 17:36:38.085936 1 data-processor.go:245] Validating image I0420 17:36:38.098500 1 data-processor.go:239] New phase: Complete I0420 17:36:38.098581 1 importer.go:212] Import Complete
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Virtualization 2.6.2 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:1502