Bug 1929519

Maybe virt-clone is the component here which needs to be fixed instead
of libvirt, could the libvirt team please confirm?

(In reply to Christian Horn from comment #1)
> Maybe virt-clone is the component here which needs to be fixed instead
> of libvirt, could the libvirt team please confirm?

Yes, we need to fix both components.

(In reply to Jaroslav Suchanek from comment #2)
> (In reply to Christian Horn from comment #1)
> > Maybe virt-clone is the component here which needs to be fixed instead
> > of libvirt, could the libvirt team please confirm?
> 
> Yes, we need to fix both components.

Thanks.  I think virt-clone is part of virt-install, component is
virt-manager.  Do we need a further bz then for virt-manager?

(In reply to Christian Horn from comment #3)
> (In reply to Jaroslav Suchanek from comment #2)
> > (In reply to Christian Horn from comment #1)
> > > Maybe virt-clone is the component here which needs to be fixed instead
> > > of libvirt, could the libvirt team please confirm?
> > 
> > Yes, we need to fix both components.
> 
> Thanks.  I think virt-clone is part of virt-install, component is
> virt-manager.  Do we need a further bz then for virt-manager?

I cloned it to bug 1935810.

Thanks.

So what happens here, is that virt-clone calls virStorageVolCreateXMLFrom() passing in the original disk image. This instructs libvirt to run 'qemu-img convert'. Meanwhile, virt-clone calls virStorageVolGetInfo() every second to fetch new allocation size. As such, the API indeed returns correct numbers but those are not interpreted correctly (the file allocation on XFS does work in chunks and may temporarily exceed desired file size; I've been meeting this problem when implementing sparse streams - see bug 1282859#c15). I will let others to express their thoughts, but meanwhile this is low priority.

Bulk update: Move RHEL-AV bugs to RHEL8.

Since the cloned from bug is a virt-manager related and this bug has a Fujitsu relationship, leave this on RHEL8. If resolution/testing is needed on RHEL9, then a RHEL9 clone can be created.

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Hm.. this here is the rhel9 clone of bz1877163 which our partner
reported for rhel8.
Seems like this one here got closed without being picked up.

So if the issue is in rhel9, while it's fixed now in rhel8, then
customers will see a regression when upgrading.

bz1877163 (rhel8) was solved by 
https://access.redhat.com/errata/RHSA-2021:4191 .
I looked at the changelog, but found no reference to bz1877163 .
That errata has libvirt-6.0.0-37.module+el8.5.0+12162+40884dd2.src.rpm.
https://bugzilla.redhat.com/show_bug.cgi?id=1877163#c65 mentions a patch,
but I found no traces of that in libvirt-6.0.0-37.module+el8.5.0+12162+40884dd2.src.rpm .
The patch is probably in though, as QA did sanity verification, in other words 
found the patch.

I will consult with the primary TAM on further steps here,
setting needinfo so this is not forgotten.

Note that libvirt didn't in any version change how the progress is being reported. I'm not sure how the virt-clone bug was solved, but it was not by adopting a different method of counting progress by libvirt.

From my comment at https://bugzilla.redhat.com/show_bug.cgi?id=1877163#c38:

> We can backport the extent size hint which will make the accidental
> approximation match reality better, but it is not a proper bug fix. It only
> hides the bug until something else changes the preallocation strategy again
> (or until you want really precise information instead of just an
> approximation).

This is what we ended up doing. The patch backported for the RHEL 8 bug is already part of RHEL 9.0 due to the rebase to a newer upstream version of QEMU.

Maybe QE could test that the behaviour is still the same in 9.1 as in 8.5?

(In reply to Kevin Wolf from comment #11)
> From my comment at https://bugzilla.redhat.com/show_bug.cgi?id=1877163#c38:
> 
> > We can backport the extent size hint which will make the accidental
> > approximation match reality better, but it is not a proper bug fix. It only
> > hides the bug until something else changes the preallocation strategy again
> > (or until you want really precise information instead of just an
> > approximation).
> 
> This is what we ended up doing. The patch backported for the RHEL 8 bug is
> already part of RHEL 9.0 due to the rebase to a newer upstream version of
> QEMU.
> 
> Maybe QE could test that the behaviour is still the same in 9.1 as in 8.5?

Hi Kevin,

Packages version:
qemu-kvm-7.0.0-12.el9.x86_64
libvirt-8.5.0-5.el9.x86_64
virt-install-4.0.0-1.el9.noarch
virt-manager-4.0.0-1.el9.noarch


Steps:

1. Prepare a shutoff VM whose filesystem is XFS.

# cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Wed Aug  3 03:14:57 2022
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/rhel-root   /                       xfs     defaults        0 0
UUID=6ab6de88-f76d-4def-8f1c-188c7b108d4c /boot                   xfs     defaults        0 0
UUID=82EB-0312          /boot/efi               vfat    umask=0077,shortname=winnt 0 2
/dev/mapper/rhel-swap   none                    swap    defaults        0 0



# qemu-img info /var/lib/libvirt/images/GuestOne.qcow2 
image: /var/lib/libvirt/images/GuestOne.qcow2
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 1.43 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: true
    refcount bits: 16
    corrupt: false
    extended l2: false



2. Clone the VM.


# virt-clone --file /var/lib/libvirt/images/GuestOne-clone.qcow2 --original GuestOne --name GuestOne-clone  --nonsparse
Allocating 'GuestOne-clone.qcow2'                                            |  20 GB  00:00:12 !!! 

Clone 'GuestOne-clone' created successfully.

# qemu-img info /var/lib/libvirt/images/GuestOne-clone.qcow2 
image: /var/lib/libvirt/images/GuestOne-clone.qcow2
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 20 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: true
    refcount bits: 16
    corrupt: false
    extended l2: false


Test result: During the cloning progress, the process bar is 100% all the time.

And Core has given feedback for this behavior in https://bugzilla.redhat.com/show_bug.cgi?id=1935810#c6 

@Kevin, Please have a look at my test, thanks.

@Core, this has been closed as WONFIX, and how about your opinion on the blocked bug 1935810, thanks.

My opinion on bug 1935810 hasn't changed. simplest 'fix' I can think of is to disable progress reporting in the --nonsparse case. But we would be trading 'virt-clone progress pegged at 100%' for 'virt-clone not reporting progress' so it's not some great improvement.