Bug 1628477 - [downstream clone - 4.2.7] After importing KVM VM the actual size is bigger than the virtual size
Summary: [downstream clone - 4.2.7] After importing KVM VM the actual size is bigger t...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 4.2.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ovirt-4.2.7
: ---
Assignee: Steven Rosenberg
QA Contact: Nisim Simsolo
URL:
Whiteboard:
Depends On: 1636042 1637410
Blocks: 1625543
TreeView+ depends on / blocked
 
Reported: 2018-09-13 08:02 UTC by RHV bug bot
Modified: 2022-03-13 15:37 UTC (History)
22 users (show)

Fixed In Version: v4.20.40
Doc Type: Enhancement
Doc Text:
When a VM is running, the disk size of the virtual machine should be no larger than was required during the initial allocation of disk space, unless you specify pre-allocation. Previously, when you set thin provisioning for importing a KVM-based VM into a Red Hat Virtualizaton environment, the disk size of the VM within the Red Hat Virtualization storage domain was inflated to the volume size or larger, even when the original KVM-based VM was much smaller. KVM Sparseness is now supported so that when you import a virtual machine with thin provisioning enabled into a Red Hat Virtualization environment, the disk size of the original virtual machine image is preserved.
Clone Of: 1625543
Environment:
Last Closed: 2018-11-05 15:02:07 UTC
oVirt Team: Virt
Target Upstream Version:
Embargoed:
lsvaty: testing_plan_complete-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-45193 0 None None None 2022-03-13 15:37:22 UTC
Red Hat Product Errata RHEA-2018:3478 0 None None None 2018-11-05 15:02:51 UTC
oVirt gerrit 94109 0 None ABANDONED virt: vm: Added Sparse support for Import 2020-08-02 12:51:09 UTC
oVirt gerrit 94110 0 None MERGED virt: vm: Added Sparse support for Import 2020-08-02 12:51:09 UTC
oVirt gerrit 94169 0 ovirt-4.2 MERGED virt: vm: Added Sparse support for Import 2020-08-02 12:51:09 UTC

Comment 8 RHV bug bot 2018-09-13 08:03:54 UTC
1. To test this feature, one needs to first set up a KVM Host. This may be performed via the Virtual Machine Manager which can connect to a remote machine for saving KVM VMs.

One can check the Volume and Disk size of the VM image file via the following command:

qemu-img info  <file Image>

One can find the image file here:

/var/lib/libvirt/images

A sample output is as follows:

[root@sla-leonard ~]# qemu-img info /var/lib/libvirt/images/generic.qcow2
image: /var/lib/libvirt/images/generic.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 3.3M
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false


2. Enter the WebAdmin oVirt Open Visualization Manager. Enter the Compute >> Virtual Machines tab. At the far right on the screen after the Create Snapshot button are three vertical dots. Please press on the button with the three vertical dots and choose Import on the drop down list. Choose the Data Center, Change the Source Drop Down List to KVM (Via Libvirt). Enter the URI, User and Password of the KVM Host and click on the Load button.

Note: at least one VM has to be running in the Data Center.

The URI should be in the following format (replacing HostName with the IP Address or Host Name of the KVM Host):

 qemu+ssh://root@HostName/system  

3. One should see the list of virtual machines available in the “Virtual Machines on Source” List Box on the left side. Please choose the VMs to be imported to oVirt, press the right arrow button so that the VMs chosen appear in the “Virtual Machines to Import” List Box on the right side and press the Next button.

4. Ensure the Allocation Policy is set to “Thin Provision” and choose the OK button.

5. Wait for the Import to succeed.

6. One can check the file on the storage domain to verify the Disk Size matches the original.

(Originally by Steven Rosenberg)

Comment 10 RHV bug bot 2018-09-13 08:04:11 UTC
flags,please

(Originally by michal.skrivanek)

Comment 11 RHV bug bot 2018-09-19 12:16:08 UTC
INFO: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Project 'vdsm'/Component 'ovirt-engine' mismatch]

For more info please contact: rhv-devops

Comment 13 Nisim Simsolo 2018-09-20 15:29:38 UTC
Using ovirt-engine-4.2.7-0.1.el7ev build, I see that for Linux VMs, the disk size is preserved:
KVM source: 
[root@alma01 ~]# qemu-img info  /var/lib/libvirt/images/nsimsolo_kvm/rhel7.2-clone.qcow2
image: /var/lib/libvirt/images/nsimsolo_kvm/rhel7.2-clone.qcow2
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 921M
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
[root@alma01 ~]# 

RHV after import:
[root@lion01 yum.repos.d]# qemu-img info  /rhev/data-center/mnt/.......
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 921M
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false
[root@lion01 yum.repos.d]# 
-------------------------------------------------------------------

But for Windows VM import, it looks like disk size is inflated after the import:

KVM source:
[root@alma01 ~]# qemu-img info  /var/lib/libvirt/images/win10_40G.qcow2
image: /var/lib/libvirt/images/win10_40G.qcow2
file format: qcow2
virtual size: 40G (42949672960 bytes)
disk size: 8.1G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
[root@alma01 ~]# 

RHV after import:
[root@lion01 yum.repos.d]# qemu-img info  /rhev/data-center/mnt/.......
file format: qcow2
virtual size: 40G (42949672960 bytes)
disk size: 40G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false
[root@lion01 yum.repos.d]# 

------------------------------------------------------------

Is that the expected behavior for Windows VMs import?

Comment 14 Steven Rosenberg 2018-09-20 16:22:30 UTC
Thank you for testing this Nisim.

The importing is performed via libvirt's sparseness support which was implemented by Michal Privoznik.

Perhaps he can check this for us.

Comment 15 Michal Privoznik 2018-09-21 06:36:26 UTC
(In reply to Nisim Simsolo from comment #13)

> Is that the expected behavior for Windows VMs import?

No. Libvirt doesn't look into the image so it doesn't know whether it's a windows image or say rhel image. It simply doesn't care. There must be something else going on:

1) a bug in vdsm/ovirt/whomever is transferring the data when it enables sparseness transfer only sometimes,
2) the underlying FS doesn't support sparse files (but since it looks like an NFS this is not the case, right?)
3) a bug in libvirt

It's easy to rule out 3) even with a single instance of libvirtd. Here are the steps:

1) ensure /rhev/data-center/mnt/ is a libvirt storage pool, e.g. via:
  virsh pool-define-as  --name nfs --type dir --target /rhev/data-center/mnt/
  virsh pool-start nfs

2) create an empty volume there:
  virsh vol-create-as --pool nfs --name image.qcow2 --capacity 40G --format qcow2

3) upload the image:
  virsh vol-upload --sparse --pool nfs --vol image.qcow2 /var/lib/libvirt/images/win10_40G.qcow2

(don't forget to alter the paths to fit your setup)


Oh, it just occurred to me - are the files freshly created or are you overwriting an existing ones? Because when overwriting a volume that is already fully allocated libvirt will not shrink it. Sparse streams aim at effective data transfer rather than creating sparse files.

Comment 16 Nisim Simsolo 2018-10-03 13:57:49 UTC
Files are freshly created.

Comment 17 Michal Skrivanek 2018-10-03 14:12:10 UTC
(In reply to Michal Privoznik from comment #15)
> (In reply to Nisim Simsolo from comment #13)
> 
> > Is that the expected behavior for Windows VMs import?
> 
> No. Libvirt doesn't look into the image so it doesn't know whether it's a
> windows image or say rhel image. It simply doesn't care. There must be
> something else going on:
> 
> 1) a bug in vdsm/ovirt/whomever is transferring the data when it enables
> sparseness transfer only sometimes,

it's done unconditionally, so shouldn't be the case

> 2) the underlying FS doesn't support sparse files (but since it looks like
> an NFS this is not the case, right?)

tht is possible. It only works with NFS 4.2, and while that is RHEL 7.5 (IIRC) we had further bugs on our storage side and RHEL side which prevented the version to actually be negotiated as 4.2
Please check that or try different storage backend.

> 3) a bug in libvirt
> 
> It's easy to rule out 3) even with a single instance of libvirtd. Here are
> the steps:
> 
> 1) ensure /rhev/data-center/mnt/ is a libvirt storage pool, e.g. via:
>   virsh pool-define-as  --name nfs --type dir --target /rhev/data-center/mnt/
>   virsh pool-start nfs
> 
> 2) create an empty volume there:
>   virsh vol-create-as --pool nfs --name image.qcow2 --capacity 40G --format
> qcow2
> 
> 3) upload the image:
>   virsh vol-upload --sparse --pool nfs --vol image.qcow2
> /var/lib/libvirt/images/win10_40G.qcow2
> 
> (don't forget to alter the paths to fit your setup)

this could be alternative check to verify it is underlying storage issue (or not)

Comment 18 Nisim Simsolo 2018-10-10 15:21:30 UTC
When importing KVM VM with NFSv4.2 on both sides and libvirt > 3.4.0, the import succeed and disk size is preseved. for example: 

KVM server: 

[root@localhost ~]# mount
XXXXX.XX.XX.tlv.redhat.com:/Compute_NFS/nsimsolo_kvm_nfs42_1 on /var/lib/libvirt/images/aaaa type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=1.2.3.4,local_lock=none,addr=5.6.7.8)
[root@localhost ~]#

[root@localhost ~]# qemu-img info /var/lib/libvirt/images/aaaa/win2k12r2.qcow2 
image: /var/lib/libvirt/images/aaaa/win2k12r2.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 7.4G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false
[root@localhost ~]# 

RHV Host:

[root@lion01 images]#mount
XXXXX.XX.XX.tlv.redhat.com:/Compute_NFS/nsimsolo_kvm_nfs42 on /rhev/data-center/mnt/xxxx-xxxx.xx.xxx.tlv.redhat.com:_Compute__NFS_nsimsolo__kvm__nfs42 type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,clientaddr=11.22.33.44,local_lock=none,addr=55.66.77.88)
[root@lion01 images]#

[root@lion01 images]# qemu-img info 2f9212f8-5a50-4ec5-8019-19a373999e78/45c4e501-26f0-48ae-aabd-d1b42b5d715d
image: 2f9212f8-5a50-4ec5-8019-19a373999e78/45c4e501-26f0-48ae-aabd-d1b42b5d715d
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 7.4G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false
[root@lion01 images]#

Comment 19 Nisim Simsolo 2018-10-23 10:24:06 UTC
Verification builds:
ovirt-engine-4.2.7.3-0.1.el7ev
vdsm-4.20.43-1.el7ev.x86_64
libvirt-client-4.5.0-10.el7_6.2.x86_64
qemu-kvm-rhev-2.12.0-18.el7_6.1.x86_64
sanlock-3.6.0-1.el7.x86_64

Verification scenario: 
Polarion TC added to external trackers.

Comment 21 errata-xmlrpc 2018-11-05 15:02:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:3478

Comment 22 Daniel Gur 2019-08-28 13:12:05 UTC
sync2jira

Comment 23 Daniel Gur 2019-08-28 13:16:17 UTC
sync2jira


Note You need to log in before you can comment on or make changes to this bug.