RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1225467 - virt-sparsify refuses to work on qcow2 files
Summary: virt-sparsify refuses to work on qcow2 files
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 7.2
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-27 12:46 UTC by David Jaša
Modified: 2023-03-24 13:31 UTC (History)
7 users (show)

Fixed In Version: libguestfs-1.28.1-1.37.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-19 07:01:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
full output of virt-sparsify -v (5.06 KB, text/plain)
2015-05-27 12:46 UTC, David Jaša
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2183 0 normal SHIPPED_LIVE libguestfs bug fix and enhancement update 2015-11-19 08:18:42 UTC

Description David Jaša 2015-05-27 12:46:18 UTC
Created attachment 1030539 [details]
full output of virt-sparsify -v

Description of problem:
virt-sparsify refuses to work on qcow2 files

Version-Release number of selected component (if applicable):
libguestfs-1.28.1-1.36.el7.x86_64
qemu-kvm-1.5.3-88.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. have a virtual disk backed by qcow2 file
2. use virt-sparsify to reduce the disk size: virt-sparsify --in-place disk.qcow2
3. boot into a VM with that disk attached as scsi disk with <driver ... discard='unmap'/> and issue there as root: fstrim -v <respective mount point>

Actual results:
2. virt-sparsify refuses to work with this error:
virt-sparsify: error: discard/trim is not supported: discard cannot be 
enabled on this drive: qemu < 1.6 cannot do discard on qcow2 files
3. qemu shrinks the qcow2 file successfully by the trimmed data size (note that fstrim -v prints all the free space in the FS, not just allocated free space).

Expected results:
virt-sparsify successfully shrinks the qcow2 disk as well

Additional info:

Comment 1 Richard W.M. Jones 2015-05-27 13:25:42 UTC
A simple reproducer is:

  $ qemu-img create -f qcow2 /tmp/test.qcow2 10M
  Formatting '/tmp/test.qcow2', fmt=qcow2 size=10485760 encryption=off cluster_size=65536 lazy_refcounts=off 
  $ guestfish add /tmp/test.qcow2 format:qcow2 discard:enable : run
  libguestfs: error: discard cannot be enabled on this drive: qemu < 1.6 cannot do discard on qcow2 files

In this case, qemu-kvm-1.5.3-86.el7.x86_64 is installed.  The error
would not happen with qemu-kvm-rhev, since the version number of
qemu-kvm-rhev is >= 1.6 in RHEL >= 7.1.

The error is caused by virt-sparsify requesting discard = "enable"
as a parameter of add_drive_opts.  Refer to the API documentation
here:

  http://libguestfs.org/guestfs.3.html#guestfs_add_drive_opts

The error only happens when the disk image format is qcow2.

What is supposed to happen is that libguestfs will use a qemu drive
parameter like:

  -drive file=...,discard=unmap

In qemu 1.5 (upstream) the qcow2 driver ignored the unmap parameter.
In qemu 1.6 (upstream) the qcow2 driver understood the unmap parameter
and can turn discard requests into host filesystem unmap operations.

The question is: does qemu-kvm-1.5.3-86.el7.x86_64 actually support qcow2
discards?  Which I'm still trying to determine ...

Comment 2 Richard W.M. Jones 2015-05-27 13:59:41 UTC
I finally was able to show that qemu-kvm-1.5.3-86.el7.x86_64
*does* support trimming of both raw and qcow2 files.  As this took
me a substantial amount of time, it's worth documenting it here:

(0) Ensure you have qemu-kvm-1.5.3 installed (not qemu-kvm-rhev).

(1) Patch out the bogus test in src/launch-direct.c and rebuild

diff --git a/src/launch-direct.c b/src/launch-direct.c
index fa3bb7a..c97e1a5 100644
--- a/src/launch-direct.c
+++ b/src/launch-direct.c
@@ -1434,7 +1434,7 @@ guestfs_int_discard_possible (guestfs_h *g, struct drive *
   /* qemu >= 1.6.  This was the first version that supported unmap on
    * qcow2 backing files.
    */
-  bool qemu16 = qemu_version >= 1006000;
+  /*  bool qemu16 = qemu_version >= 1006000;*/
 
   if (!qemu15)
     NOT_SUPPORTED (g, false,
@@ -1460,10 +1460,12 @@ guestfs_int_discard_possible (guestfs_h *g, struct drive
   else if (STREQ (drv->src.format, "raw"))
     /* OK */ ;
   else if (STREQ (drv->src.format, "qcow2")) {
+    /*
     if (!qemu16)
       NOT_SUPPORTED (g, false,
                      _("discard cannot be enabled on this drive: "
                        "qemu < 1.6 cannot do discard on qcow2 files"));
+    */
   }
   else {
     /* It's possible in future other formats will support discard, but

(2) Copy tests/discard/*.pl to /tmp.  *NB* These tests do not work on NFS.

(3) Run the tests like this:

$ ~/d/libguestfs-rhel-7.2/run ./test-blkdiscard.pl
$ ~/d/libguestfs-rhel-7.2/run ./test-discard.pl
$ ~/d/libguestfs-rhel-7.2/run ./test-fstrim.pl

and observe the output.  This shows that *raw* format files can be unmapped.

(4) Edit the '$format = ...' line in each test to "qcow2".

(5) Run the tests again as above.  This shows that *qcow2* format files
can be unmapped.

Since this error message turns out to be bogus, we should remove it
from RHEL 7, hence I am dev-ack-ing this bug.

Comment 3 Richard W.M. Jones 2015-05-27 14:44:52 UTC
(In reply to Richard W.M. Jones from comment #2)
> (2) Copy tests/discard/*.pl to /tmp.  *NB* These tests do not work on NFS.

Apparently the NFS problem will be fixed in 7.2 (bug 1079385).

Comment 5 Hu Zhang 2015-09-13 10:57:48 UTC
Verified with the packages:
libguestfs-1.28.1-1.52.el7.x86_64
qemu-kvm-1.5.3-102.el7.x86_64

Verify steps:
1. # qemu-img create -f qcow2 /tmp/test.qcow2 10M
Formatting '/tmp/test.qcow2', fmt=qcow2 size=10485760 encryption=off cluster_size=65536 lazy_refcounts=off

2. # guestfish add /tmp/test.qcow2 format:qcow2 discard:enable : run -x -v
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_pgroup true
libguestfs: trace: set_pgroup = 0
libguestfs: trace: add_drive "/tmp/test.qcow2" "format:qcow2" "discard:enable"
libguestfs: trace: add_drive = 0
libguestfs: trace: launch
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version *>
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "libvirt"
libguestfs: launch: program=guestfish
...
...
verbose daemon enabled
linux command line: panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color

trying to open virtio-serial channel '/dev/virtio-ports/org.libguestfs.channel.0'
udevadm --debug settle
calling: settle
libguestfs: recv_from_daemon: received GUESTFS_LAUNCH_FLAG
libguestfs: [03399ms] appliance is up
libguestfs: trace: launch = 0
libguestfs: trace: shutdown
libguestfs: trace: internal_autosync
guestfsd: main_loop: new request, len 0x28
umount-all: /proc/mounts: fsname=rootfs dir=/ type=rootfs opts=rw freq=0 passno=0
umount-all: /proc/mounts: fsname=proc dir=/proc type=proc opts=rw,relatime freq=0 passno=0
umount-all: /proc/mounts: fsname=/dev/root dir=/ type=ext2 opts=rw,noatime freq=0 passno=0
umount-all: /proc/mounts: fsname=/proc dir=/proc type=proc opts=rw,relatime freq=0 passno=0
umount-all: /proc/mounts: fsname=/sys dir=/sys type=sysfs opts=rw,relatime freq=0 passno=0
umount-all: /proc/mounts: fsname=tmpfs dir=/run type=tmpfs opts=rw,nosuid,relatime,size=97716k,mode=755 freq=0 passno=0
umount-all: /proc/mounts: fsname=/dev dir=/dev type=devtmpfs opts=rw,relatime,size=241840k,nr_inodes=60460,mode=755 freq=0 passno=0
fsync /dev/sda
libguestfs: trace: internal_autosync = 0
libguestfs: calling virDomainDestroy "guestfs-bsg220q5cfq3uncw" flags=VIR_DOMAIN_DESTROY_GRACEFUL
libguestfs: trace: shutdown = 0
libguestfs: trace: close
libguestfs: closing guestfs handle 0x7fddff73b470 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsOindop

No errors pop up. So verified.

Comment 7 errata-xmlrpc 2015-11-19 07:01:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2183.html


Note You need to log in before you can comment on or make changes to this bug.