RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1845518 - qemu-img convert to preallocated image makes it sparse
Summary: qemu-img convert to preallocated image makes it sparse
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: virt-v2v
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: beta
: ---
Assignee: Virtualization Maintenance
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 1791831 (view as bug list)
Depends On:
Blocks: 1897025
TreeView+ depends on / blocked
 
Reported: 2020-06-09 12:34 UTC by Xiaodai Wang
Modified: 2022-05-16 09:58 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-16 09:58:39 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Xiaodai Wang 2020-06-09 12:34:18 UTC 
Description of problem:
qemu-img convert to preallocated image makes it sparse

Version-Release number of selected component (if applicable):
virt-v2v-1.42.0-4.module+el8.3.0+6798+ad6e66be.x86_64
libguestfs-1.42.0-2.module+el8.3.0+6798+ad6e66be.x86_64
qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64
libvirt-6.4.0-1.module+el8.3.0+6881+88468c00.x86_64
nbdkit-1.20.2-1.module+el8.3.0+6764+cc503f20.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Convert a qcow2 sparsed disk to a preallocated image.
virt-v2v -i libvirt -ic xen+ssh://root.x.x/ xen-hvm-rhel7.8-x86_64 -o libvirt -os v2v_dir -oa preallocated -b virbr0 -n default -v -x -of qcow2
2. Check the disk format after v2v runs success.

Actual results:
2020-06-05 05:14:03,466 process          L0458 DEBUG| [stdout] image: /var/tmp/avocado_ffxs145o/v2v_dir_pool/xen-hvm-rhel7.8-x86_64-sda
2020-06-05 05:14:03,466 process          L0714 INFO | Command 'qemu-img info /var/tmp/avocado_ffxs145o/v2v_dir_pool/xen-hvm-rhel7.8-x86_64-sda -U' finished with 0 after 0.004693508148193359s
2020-06-05 05:14:03,466 process          L0458 DEBUG| [stdout] file format: qcow2
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout] virtual size: 8.79 GiB (9437184000 bytes)
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout] disk size: 4.97 GiB
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout] cluster_size: 65536
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout] Format specific information:
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout]     compat: 1.1
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout]     lazy refcounts: false
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout]     refcount bits: 16
2020-06-05 05:14:03,467 process          L0458 DEBUG| [stdout]     corrupt: false

Expected results:
The disk size should be not less than virtual size.

Additional info:
This a upstream discussion about this problem.
https://www.mail-archive.com/qemu-devel@nongnu.org/msg671142.html
Comment 1 Richard W.M. Jones 2020-06-09 12:48:25 UTC 
This problem turned up in the virt-v2v test suite around the end of
last year.  To summarise what's happening briefly: When using the -oa preallocated
option we want to create a fully allocated guest.  To do this we
do something like this:

(a) qemu-img create [the output] -o preallocation=falloc,compat=1.1

(b) qemu-img convert [the source] [the output]

In versions of qemu before commit 4d7c487eac1652dfe4498fe84f32900ad461d61b
the copy operation (b) would preserve the fully allocated state of
the output.  However after this commit qemu-img actually trims the
output when it sees sparseness in the input, making the output sparse.

Upstream added a --target-is-zero option which I believe can be used to
fix this.  However we mustn't add it unconditionally.  There are some
corner cases where we cannot guarantee that the target is really zero,
eg. if writing to a block device.
Comment 2 Richard W.M. Jones 2020-06-09 12:53:56 UTC 
I have confirmed that --target-is-zero does fix the example that
I gave in the qemu-devel mailing list link above.
Comment 3 Richard W.M. Jones 2020-06-09 13:45:49 UTC 
*** Bug 1791831 has been marked as a duplicate of this bug. ***
Comment 5 Richard W.M. Jones 2021-04-27 16:21:00 UTC 
The long term plan to fix this is by using nbdcopy.  See:
https://listman.redhat.com/archives/libguestfs/2020-November/msg00022.html
https://bugzilla.redhat.com/show_bug.cgi?id=1798105
Comment 6 xinyli 2021-06-03 09:02:36 UTC 
I can reproduce this  bug when using virt-v2v to convert guest to RHV export domain.
Version:
virt-v2v-1.44.0-1.el9.1.x86_64
libguestfs-1.45.6-1.el9.x86_64
qemu-kvm-6.0.0-2.el9.x86_64
libvirt-7.3.0-1.el9.x86_64
nbdkit-1.25.7-1.el9.x86_64
Steps:
1. Convert a guest with raw sparse disk using -oa preallocated -of raw.
# virt-v2v -i libvirt rhel8-raw -o rhv -os 10.73.224.195:/home/nfs_export --network ovirtmgmt  -oa preallocated -of raw
[   0.0] Opening the source -i libvirt rhel8-raw
[   0.0] Creating an overlay to protect the source from being modified
[   0.1] Opening the overlay
[   4.5] Inspecting the overlay
[   7.7] Checking for sufficient free disk space in the guest
[   7.7] Estimating space required on target for each disk
[   7.7] Converting Red Hat Enterprise Linux 8.5 Beta (Ootpa) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  59.2] Mapping filesystem data to avoid copying unused and blank areas
[  60.2] Closing the overlay
[  60.5] Assigning disks to buses
[  60.5] Checking if the guest needs BIOS or UEFI to boot
[  60.5] Initializing the target -o rhv -os 10.73.224.195:/home/nfs_export
[  60.7] Copying disk 1/1 to /tmp/v2v.ewvDSB/91804183-2de6-48ab-a6fb-e82d501a36c7/images/763f550e-8411-40f8-94ea-8b1ed3480994/87e4802f-f4f6-4b53-8bd5-4061dd07beca (raw)
    (100.00/100%)
[ 126.8] Creating output metadata
virt-v2v: warning: oVirt cannot emulate ‘ich9’ sound cards.  This sound 
card will be dropped from the output.
[ 126.9] Finishing off

2. After v2v runs successfully, mount RHV export domain to check the disk format.
#mount 10.73.224.195:/home/nfs_export /mnt/
# qemu-img info /mnt/91804183-2de6-48ab-a6fb-e82d501a36c7/images/763f550e-8411-40f8-94ea-8b1ed3480994/87e4802f-f4f6-4b53-8bd5-4061dd07beca
image: /mnt/91804183-2de6-48ab-a6fb-e82d501a36c7/images/763f550e-8411-40f8-94ea-8b1ed3480994/87e4802f-f4f6-4b53-8bd5-4061dd07beca
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 2.86 GiB

3.Check disk metadata
# cat /mnt/91804183-2de6-48ab-a6fb-e82d501a36c7/images/763f550e-8411-40f8-94ea-8b1ed3480994/87e4802f-f4f6-4b53-8bd5-4061dd07beca.meta 
DOMAIN=91804183-2de6-48ab-a6fb-e82d501a36c7
VOLTYPE=LEAF
CTIME=1622708875
MTIME=1622708875
IMAGE=763f550e-8411-40f8-94ea-8b1ed3480994
DISKTYPE=2
PUUID=00000000-0000-0000-0000-000000000000
LEGALITY=LEGAL
POOL_UUID=
SIZE=20971520
FORMAT=RAW
TYPE=PREALLOCATED
DESCRIPTION=generated by virt-v2v 1.44.0rhel_9,release_1.el9.1
EOF
Comment 10 RHEL Program Management 2021-12-09 07:27:12 UTC 
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
Comment 11 Richard W.M. Jones 2021-12-09 11:51:37 UTC 
This bug was closed in error by a process that we have no control over.
I am reopening it.

Xiaodai - this should be fixed by virt-v2v >= 1.45.9x
Comment 12 Laszlo Ersek 2022-05-13 13:15:29 UTC 
Can you please retest -- the latest virt-v2v uses nbdcopy and should not have this problem (per comment 5 and comment 11).
Comment 13 Xiaodai Wang 2022-05-16 09:43:53 UTC 
I tried the conversion of this bug in RHEL9.1, the disk size is correct now.

# qemu-img info xen-hvm-rhel7.8-x86_64-sda 
image: xen-hvm-rhel7.8-x86_64-sda
file format: qcow2
virtual size: 8.79 GiB (9437184000 bytes)
disk size: 8.79 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false

and nbdcopy is used during copying process.

so I think this bug can be closed as CURRENTRELEASE.
Comment 14 Richard W.M. Jones 2022-05-16 09:58:39 UTC 
Thanks for retesting this.  I'm closing currentrelease per comment 13.
Note You need to log in before you can comment on or make changes to this bug.