Bug 1946177

+++ This bug was initially created as a clone of Bug #1945121 +++

Created attachment 1768054 [details]
Pod's log with errors

Description of problem: Migration from RHV fails if using NFS as a target storage


Version-Release number of selected component (if applicable): CNV 2.6.1


How reproducible: 100%


Steps to Reproduce:
1. Start migration wizard as usual
2. When defining target storage - use NFS with filesystem
3. Start migration and take a look on import log

Actual results: Migration will fail on resizing target image


Expected results: Migration succeeded


Additional info:

I0331 08:51:16.058986       1 data-processor.go:239] New phase: Resize
W0331 08:51:16.423118       1 data-processor.go:331] Available space less than requested size, resizing image to available space 4281974784.
I0331 08:51:16.423155       1 data-processor.go:337] Expanding image size to: 4281974784
E0331 08:51:16.438761       1 prlimit.go:174] qemu-img failed output is:
E0331 08:51:16.438789       1 prlimit.go:175] 
E0331 08:51:16.438796       1 prlimit.go:176] qemu-img: Use the --shrink option to perform a shrink operation.
qemu-img: warning: Shrinking an image will delete all data beyond the shrunken image's end. Before performing such an operation, make sure there is no important data there.
E0331 08:51:16.438856       1 data-processor.go:236] exit status 1
qemu-img execution failed

--- Additional comment from Amos Mastbaum on 2021-03-31 12:18:08 UTC ---

Same behavior with cephs/Filesystem (while cephs/block has no issues)
Maybe this is the issue.

--- Additional comment from Amos Mastbaum on 2021-03-31 12:18:25 UTC ---

Same behavior with cephs/Filesystem (while cephs/block has no issues)
Maybe this is the issue.

--- Additional comment from Maya Rashish on 2021-03-31 14:24:02 UTC ---

I backported the PR that introduced this breakage to CDI: https://github.com/kubevirt/containerized-data-importer/pull/1612
I am happy to rip it out of 2.6.1 if it comes to that, there is no strong reason to backport it.
But that will push out the problem to 4.8, not make it go away.

--- Additional comment from Alexander Wels on 2021-03-31 18:58:00 UTC ---

Can I get a copy of the disk image that is written to the PVC? The PVC is already converted, but not yet resized. That way I can use it to track down the problem. Also I would like a copy of the DV that creates the PVC.

--- Additional comment from Ilanit Stein on 2021-03-31 19:51:29 UTC ---



--- Additional comment from Maya Rashish on 2021-04-01 16:06:58 UTC ---

This is the workaround:

oc patch cdi cdi-kubevirt-hyperconverged --type merge -p '{"spec": {"config": {"filesystemOverhead": {"global": "0"}}}}'

This will unset the value:

oc patch cdi cdi-kubevirt-hyperconverged --type merge -p '{"spec": {"config": {"filesystemOverhead": {}}}}'


Does this work around work?

--- Additional comment from Ilanit Stein on 2021-04-01 17:58:51 UTC ---

Thanks Maya,

The work around works fine. 

I used a source VM fedora32 4G disk and run those steps:

1. Without running the workaround applied yet, 
VM import from RHV to destination storage NFS/Filesystem-> Failed at the resize phase, as expected.

2. Apply the workaround

3. VM import from RHV to destination storage NFS/Filesystem ->PASS, VM started OK.

4. VM import from RHV to destination storage Ceph-RBD/Block ->PASS.

5. Run the command to unset the value - If here the intention was to return the behavior to how it was before applying the work around then this didn't work. VM import from RHV to NFS still PASSES. 
(Looks like the resize phase is still not running, but I didn't catch the exact log last message on this). 

Maya, 
Can you advise on this?

--- Additional comment from Maya Rashish on 2021-04-01 19:19:48 UTC ---

Add:
oc patch cdi cdi-kubevirt-hyperconverged --type merge -p '{"spec": {"config": {"filesystemOverhead": {"global": "0"}}}}'

Remove:
oc patch cdi cdi-kubevirt-hyperconverged --type=json -p '[{ "op": "remove", "path": "/spec/config/filesystemOverhead" }]'