Bug 1882917
| Summary: | the target image size is incorrect when converting a badly fragmented file | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Xueqiang Wei <xuwei> |
| Component: | qemu-kvm | Assignee: | Kevin Wolf <kwolf> |
| qemu-kvm sub component: | Storage | QA Contact: | Tingting Mao <timao> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | chayang, coli, hreitz, jinzhao, juzhang, kwolf, mrezanin, qzhang, virt-maint, xuwei, yama |
| Version: | 9.0 | Keywords: | Triaged |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | qemu-kvm-6.2.0-8.el9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-05-17 12:23:22 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Tested on rhel8.2.1-av, also hit similar issue.
Versions:
kernel-4.18.0-234.el8.x86_64
qemu-kvm-4.2.0-29.module+el8.2.1+7990+27f1e480.4
# qemu-img create -f raw test.raw 10G
# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192)
Run completed in 51.650 seconds.
# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192)
Run completed in 51.120 seconds.
# filefrag test.raw
test.raw: 2000000 extents found
# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw
(100.00/100%)
# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2
(100.00/100%)
# qemu-img info test.raw
image: test.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.66 GiB
# qemu-img info convert.raw
image: convert.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 11.3 MiB
# qemu-img info convert.qcow2
image: convert.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 181 MiB
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
As far as I can tell, this was broken in commit 8dcd3c9b91a (first released in QEMU 3.0.0). Move RHEL-AV bugs to RHEL9. If necessary to resolve in RHEL8, then clone to the current RHEL8 release. QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass. Verified this bug as below:
Tested with:
qemu-kvm-6.2.0-8.el9
kernel-5.14.0-54.kpq0.el9.x86_64
Steps:
# qemu-img create -f raw test.raw 10G
# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192)
Run completed in 18.606 seconds.
# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096
Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192)
Run completed in 17.643 seconds.
# filefrag test.raw
test.raw: 7326 extents found
# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw
(100.00/100%)
# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2
(100.00/100%)
# qemu-img info test.raw
image: test.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 7.63 GiB
# qemu-img info convert.raw
image: convert.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 1 MiB
# qemu-img info convert.qcow2
image: convert.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
Results:
As above, there is no the real data in the converted images.
According to comment8, set this bug as verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: qemu-kvm), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:2307 |
Description of problem: The target image size is incorrect when converting a badly fragmented file. The target image size isn't 0 when the source image that is fully allocated with zero bytes. According to Kevin's feedback, qemu-img convert should detect the zeroes and keep the target image sparse. Version-Release number of selected component (if applicable): kernel-4.18.0-234.el8.x86_64 qemu-kvm-5.1.0-9.module+el8.3.0+8182+ac9ced32 How reproducible: 5/5 Steps to Reproduce: (Tested it on XFS file system) 1. create an image, size=10G # qemu-img create -f raw -o extent_size_hint=1M test.raw 10G 2. Create the badly fragmented file with qemu-img bench. Don't specify --pattern, write zero bytes. # qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192) Run completed in 26.615 seconds. # qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192) Run completed in 21.748 seconds. 3. check file fragmentation # filefrag test.raw test.raw: 7380 extents found 4. convert test.raw to convert.raw # qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw 5. convert test.raw to convert.qcow2 # qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2 6. check image size # qemu-img info test.raw image: test.raw file format: raw virtual size: 10 GiB (10737418240 bytes) disk size: 7.63 GiB # qemu-img info convert.raw image: convert.raw file format: raw virtual size: 10 GiB (10737418240 bytes) disk size: 2.82 GiB # qemu-img info convert.qcow2 image: convert.qcow2 file format: qcow2 virtual size: 10 GiB (10737418240 bytes) disk size: 182 MiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false Actual results: After step 6, the target image size isn't 0 Expected results: Since the source image that is fully allocated with zero bytes, the target image size should be 0. Additional info: 1. write the badly fragmented file with --pattern=1, the size is as expected. Details: (1)# qemu-img create -f raw -o extent_size_hint=1M test.raw 10G (2)# qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 0 --pattern=1 # qemu-img bench -f raw -t none -n -w test.raw -c 1000000 -S 8192 -o 4096 --pattern=1 (3)# filefrag test.raw test.raw: 7408 extents found (4)# qemu-img convert -p -f raw -O raw -T none -t none test.raw convert.raw (5)# qemu-img convert -p -f raw -O qcow2 -T none -t none test.raw convert.qcow2 (6)# qemu-img info test.raw image: test.raw file format: raw virtual size: 10 GiB (10737418240 bytes) disk size: 7.63 GiB # qemu-img info convert.raw image: convert.raw file format: raw virtual size: 10 GiB (10737418240 bytes) disk size: 7.63 GiB # qemu-img info convert.qcow2 image: convert.qcow2 file format: qcow2 virtual size: 10 GiB (10737418240 bytes) disk size: 7.63 GiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false 2. Tested on a unfragmented file, all work well. 2.1 Tested with -P0: # qemu-img create -f raw base.raw 1G # qemu-io -c 'write -P0 0 200M' base.raw -f raw wrote 209715200/209715200 bytes at offset 0 200 MiB, 1 ops; 00.13 sec (1.451 GiB/sec and 7.4285 ops/sec) # qemu-img convert -p -f raw -O raw -T none -t none base.raw base_convert.raw (100.00/100%) # qemu-img convert -p -f raw -O qcow2 -T none -t none base.raw base_convert.qcow2 (100.00/100%) # qemu-img info base.raw image: base.raw file format: raw virtual size: 1 GiB (1073741824 bytes) disk size: 200 MiB # qemu-img info base_convert.raw image: base_convert.raw file format: raw virtual size: 1 GiB (1073741824 bytes) disk size: 1 MiB # qemu-img info base_convert.qcow2 image: base_convert.qcow2 file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: 196 KiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false 2.2 Tested with -P1 # qemu-img create -f raw base.raw 1G # qemu-io -c 'write -P1 0 200M' base.raw -f raw # qemu-img convert -p -f raw -O raw -T none -t none base.raw base_convert.raw # qemu-img convert -p -f raw -O qcow2 -T none -t none base.raw base_convert.qcow2 # qemu-img info base.raw image: base.raw file format: raw virtual size: 1 GiB (1073741824 bytes) disk size: 200 MiB # qemu-img info base_convert.raw image: base_convert.raw file format: raw virtual size: 1 GiB (1073741824 bytes) disk size: 200 MiB # qemu-img info base_convert.qcow2 image: base_convert.qcow2 file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: 200 MiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false