Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1882917

Summary: the target image size is incorrect when converting a badly fragmented file
Product: Red Hat Enterprise Linux 9 Reporter: Xueqiang Wei <xuwei>
Component: qemu-kvmAssignee: Kevin Wolf <kwolf>
qemu-kvm sub component: Storage QA Contact: Tingting Mao <timao>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: chayang, coli, hreitz, jinzhao, juzhang, kwolf, mrezanin, qzhang, virt-maint, xuwei, yama
Version: 9.0Keywords: Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-6.2.0-8.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-17 12:23:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Xueqiang Wei 2020-09-26 13:15:42 UTC 
Description of problem:

The target image size is incorrect when converting a badly fragmented file.

The target image size isn't 0 when the source image that is fully allocated with zero bytes.

According to Kevin's feedback, qemu-img convert should detect the zeroes and keep the target image sparse.



Version-Release number of selected component (if applicable):
kernel-4.18.0-234.el8.x86_64
qemu-kvm-5.1.0-9.module+el8.3.0+8182+ac9ced32


How reproducible:
5/5


Steps to Reproduce: (Tested it on XFS file system)
1. create an image, size=10G
# qemu-img create -f raw -o extent_size_hint=1M test.raw 10G

2. Create the badly fragmented file with qemu-img bench. Don't specify --pattern, write zero bytes. 
# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192)
Run completed in 26.615 seconds.

# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192)
Run completed in 21.748 seconds.


3. check file fragmentation
# filefrag test.raw 
test.raw: 7380 extents found


4. convert test.raw to convert.raw
# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw

5. convert test.raw to convert.qcow2
# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2

6. check image size
# qemu-img info test.raw
image: test.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.63 GiB

# qemu-img info convert.raw
image: convert.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 2.82 GiB

# qemu-img info convert.qcow2
image: convert.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 182 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false



Actual results:
After step 6, the target image size isn't 0


Expected results:
Since the source image that is fully allocated with zero bytes, the target image size should be 0. 



Additional info:
1. write the badly fragmented file with --pattern=1, the size is as expected.

Details:
(1)# qemu-img create -f raw -o extent_size_hint=1M test.raw 10G
(2)# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0 --pattern=1
# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096 --pattern=1
(3)# filefrag test.raw 
test.raw: 7408 extents found
(4)# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw
(5)# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2
(6)# qemu-img info test.raw 
image: test.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.63 GiB

# qemu-img info convert.raw 
image: convert.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.63 GiB

# qemu-img info convert.qcow2 
image: convert.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.63 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false


2. Tested on a unfragmented file, all work well.
2.1 Tested with -P0:
# qemu-img create -f raw base.raw 1G
# qemu-io -c 'write -P0 0 200M' base.raw  -f raw
wrote 209715200/209715200 bytes at offset 0
200 MiB, 1 ops; 00.13 sec (1.451 GiB/sec and 7.4285 ops/sec)

# qemu-img convert -p -f raw -O raw -T none -t none base.raw base_convert.raw
    (100.00/100%)
# qemu-img convert -p -f raw -O qcow2 -T none -t none base.raw base_convert.qcow2
    (100.00/100%)

# qemu-img info base.raw 
image: base.raw
file format: raw
virtual size: 1 GiB (1073741824 bytes)
disk size: 200 MiB

# qemu-img info base_convert.raw 
image: base_convert.raw
file format: raw
virtual size: 1 GiB (1073741824 bytes)
disk size: 1 MiB

# qemu-img info base_convert.qcow2 
image: base_convert.qcow2
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

2.2 Tested with -P1
# qemu-img create -f raw base.raw 1G
# qemu-io -c 'write -P1 0 200M' base.raw  -f raw
# qemu-img convert -p -f raw -O raw -T none -t none base.raw base_convert.raw
# qemu-img convert -p -f raw -O qcow2 -T none -t none base.raw base_convert.qcow2

# qemu-img info base.raw
image: base.raw
file format: raw
virtual size: 1 GiB (1073741824 bytes)
disk size: 200 MiB

# qemu-img info base_convert.raw
image: base_convert.raw
file format: raw
virtual size: 1 GiB (1073741824 bytes)
disk size: 200 MiB

# qemu-img info base_convert.qcow2
image: base_convert.qcow2
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 200 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
Comment 1 Xueqiang Wei 2020-09-26 13:33:13 UTC 
Tested on rhel8.2.1-av, also hit similar issue.

Versions:
kernel-4.18.0-234.el8.x86_64
qemu-kvm-4.2.0-29.module+el8.2.1+7990+27f1e480.4


# qemu-img create -f raw test.raw 10G

# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192)
Run completed in 51.650 seconds.

# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192)
Run completed in 51.120 seconds.

# filefrag test.raw 
test.raw: 2000000 extents found

# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw
    (100.00/100%)

# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2
    (100.00/100%)

# qemu-img info test.raw 
image: test.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.66 GiB

# qemu-img info convert.raw 
image: convert.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 11.3 MiB

# qemu-img info convert.qcow2 
image: convert.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 181 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
Comment 2 Kevin Wolf 2020-09-28 08:23:35 UTC 
As far as I can tell, this was broken in commit 8dcd3c9b91a (first released in QEMU 3.0.0).
Comment 5 John Ferlan 2021-09-08 21:38:51 UTC 
Move RHEL-AV bugs to RHEL9. If necessary to resolve in RHEL8, then clone to the current RHEL8 release.
Comment 7 Yanan Fu 2022-02-15 06:19:21 UTC 
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.
Comment 8 Tingting Mao 2022-02-15 07:13:12 UTC 
Verified this bug as below:


Tested with:
qemu-kvm-6.2.0-8.el9
kernel-5.14.0-54.kpq0.el9.x86_64


Steps:
# qemu-img create -f raw test.raw 10G

# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192)
Run completed in 18.606 seconds.

# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192)
Run completed in 17.643 seconds.

# filefrag test.raw 
test.raw: 7326 extents found

# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw
    (100.00/100%)

# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2
    (100.00/100%)

# qemu-img info test.raw 
image: test.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.63 GiB

# qemu-img info convert.raw 
image: convert.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 1 MiB

# qemu-img info convert.qcow2 
image: convert.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false


Results:
As above, there is no the real data in the converted images.
Comment 12 Tingting Mao 2022-02-16 11:24:58 UTC 
According to comment8, set this bug as verified.
Comment 14 errata-xmlrpc 2022-05-17 12:23:22 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: qemu-kvm), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2307