Bug 1730955

Summary: Overlay with compat=0.10 will not be emptied after commitment
Product: Red Hat Enterprise Linux 8 Reporter: Tingting Mao <timao>
Component: qemu-kvmAssignee: Hanna Czenczek <hreitz>
Status: CLOSED CURRENTRELEASE QA Contact: Tingting Mao <timao>
Severity: low Docs Contact:
Priority: low    
Version: 8.1CC: coli, jinzhao, juzhang, ngu, qzhang, rbalakri, virt-maint, zhenyzha
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-27 11:47:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tingting Mao 2019-07-18 03:05:54 UTC
Description of problem:
As subject.


Version-Release number of selected component (if applicable):
qemu-kvm-2.12.0-81.module+el8.1.0+3619+dfe1ae01
kernel-4.18.0-112.el8


How reproducible:
100%


Steps to Reproduce:
1. Create a base image
# qemu-img create -f raw test.img 2G

2. Create snapshot with compat=0.10 based on the base image
# qemu-img create -f qcow2 -o compat=0.10 -b test.img sn.qcow2

3. Write data to the snapshot file
# qemu-io -c 'write -P 1 0 512M' sn.qcow2

4. Check info of the images
]# qemu-img info sn.qcow2 --backing-chain
image: sn.qcow2
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 512M
cluster_size: 65536
backing file: test.img
Format specific information:
    compat: 0.10
    refcount bits: 16

image: test.img
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 0

5. Commit the snapshot file
# qemu-img commit -f qcow2 sn.qcow2 -p
    (100.00/100%)
Image committed.

6. Check the info of the images again
# qemu-img info sn.qcow2 --backing-chain
image: sn.qcow2
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 512M ------------------------------- Not emptied!
cluster_size: 65536
backing file: test.img
Format specific information:
    compat: 0.10
    refcount bits: 16

image: test.img
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 512M


Actual results:
As above. The overlay has not been emptied.


Expected results:
The overlay was emptied.


Additional info:

Comment 1 Tingting Mao 2019-07-18 03:26:13 UTC
In RHEL7.7, there is no the bug, so I mark 'regression', and set priority as 'high'.


Tested with:
qemu-kvm-rhev-2.12.0-30.el7


Steps:
# qemu-img create -f raw test.img 1g

# qemu-img create -f qcow2 -b test.img sn.qcow2 -o compat=0.10

# qemu-io -c 'write -P 1 0 512m' sn.qcow2

# qemu-img info sn.qcow2 --backing-chain
image: sn.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 512M
cluster_size: 65536
backing file: test.img
Format specific information:
    compat: 0.10
    refcount bits: 16

image: test.img
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 0

# qemu-img commit -f qcow2 sn.qcow2 -p

# qemu-img info sn.qcow2 --backing-chain
image: sn.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 324K ------------------------------------- Emptied!
cluster_size: 65536
backing file: test.img
Format specific information:
    compat: 0.10
    refcount bits: 16

image: test.img
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 512M

Comment 3 Ademar Reis 2019-07-26 20:23:55 UTC
Low priority because of compat=0.10 and being a corner-case.

If not a low-hanging fruit, I recommend closing it.

Comment 4 Hanna Czenczek 2019-07-27 11:47:09 UTC
Hi,

qemu supports a fast path for emptying compat=1.1 images (with certain restrictions), and a slow path for all other images.  Only the fast path is sure to reduce the image to a minimal size.  The slow path just discards all active clusters, which may or may not have an immediate, measurable effect.

I don’t know what changed in RHEL 8 when compared to 7.7, but I don’t think it was anything in qemu.  As far as I can see from adding some debug output, we still discard all clusters in an compat=0.10 image, it just somehow doesn’t have an effect on the host.

(If I test RHEL 8’s qemu-img on a Fedora 30 host, I actually don’t see the problem.  The compat=0.10 overlay was emptied during the commit.  I tested upstream before, and I saw the issue, kind of.  The disk usage was reduced by a bit, but not by much.  If I test upstream now, the issue has disappeared.  So maybe it has something to do whether the filesystem feels like actually freeing the blocks immediately.  I suppose it could be due to XFS’s speculative preallocation, so it keeps some blocks allocated to the file and will free them only once it sees the need to.)

Closing the bug, because it looks to me like qemu still discards all blocks, as it should; I suppose the filesystem just doesn’t respond to it visibly immediately.  Please reopen the bug if your filesystem is not XFS.

Max

Comment 5 Tingting Mao 2019-07-27 14:51:43 UTC
I can not reproduce this bug in the latest qemu package even with XFS, either. 


Tested with:
qemu-kvm-2.12.0-82.module+el8.1.0+3738+0d8c0249
4.18.0-112.el8.x86_64
# df -T sn.qcow2 
Filesystem                             Type 1K-blocks      Used Available Use% Mounted on
/dev/mapper/rhel_hp--dl388pg8--01-home xfs  222836224 120377864 102458360  55% /home


Steps:
# qemu-img create -f raw test.img 2G

# qemu-img create -f qcow2 -o compat=0.10 -b test.img sn.qcow2

# qemu-io -c 'write -P 1 0 512M' sn.qcow2

# qemu-img info sn.qcow2 --backing-chain
image: sn.qcow2
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 512M
cluster_size: 65536
backing file: test.img
Format specific information:
    compat: 0.10
    refcount bits: 16

image: test.img
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 0

# qemu-img commit -f qcow2 sn.qcow2 -p
    (100.00/100%)
Image committed.

# qemu-img info sn.qcow2 --backing-chain
image: sn.qcow2
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 452K    --------------- Emptied !
cluster_size: 65536
backing file: test.img
Format specific information:
    compat: 0.10
    refcount bits: 16

image: test.img
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 512M