Bug 1166481 - Allow qemu-img to bypass the host cache (check, compare, convert, rebase, amend)
Summary: Allow qemu-img to bypass the host cache (check, compare, convert, rebase, amend)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Max Reitz
QA Contact: Virtualization Bugs
URL:
Whiteboard: storage
Depends On: 1116558 1138691
Blocks: 1138690
TreeView+ depends on / blocked
 
Reported: 2014-11-21 05:02 UTC by Sibiao Luo
Modified: 2015-03-05 09:58 UTC (History)
29 users (show)

Fixed In Version: qemu-kvm-rhev-2.1.2-14.el7
Doc Type: Rebase: Bug Fixes and Enhancements
Doc Text:
Clone Of: 1138691
Environment:
Last Closed: 2015-03-05 09:58:35 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0624 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2015-03-05 14:37:36 UTC

Comment 1 Sibiao Luo 2014-11-21 05:03:33 UTC
host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-205.el7.x86_64
qemu-kvm-rhev-2.1.2-8.el7.x86_64

# qemu-img check image-1.qcow2 -T none
check: invalid option -- 'T'
qemu-img version 2.1.2, Copyright (c) 2004-2008 Fabrice Bellard
usage: qemu-img command [command options]
QEMU disk image utility

Command syntax:
  check [-q] [-f fmt] [--output=ofmt]  [-r [leaks | all]] filename
  create [-q] [-f fmt] [-o options] filename [size]
  commit [-q] [-f fmt] [-t cache] filename
  compare [-f fmt] [-F fmt] [-p] [-q] [-s] filename1 filename2
  convert [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-O output_fmt] [-o options] [-s snapshot_id_or_name] [-l snapshot_param] [-S sparse_size] filename [filename2 [...]] output_filename
  info [-f fmt] [--output=ofmt] [--backing-chain] filename
  map [-f fmt] [--output=ofmt] filename
  snapshot [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename
  rebase [-q] [-f fmt] [-t cache] [-p] [-u] -b backing_file [-F backing_fmt] filename
  resize [-q] filename [+ | -]size
  amend [-q] [-f fmt] -o options filename
......

Comment 2 Miroslav Rezanina 2014-11-27 12:54:42 UTC
Fix included in qemu-kvm-rhev-2.1.2-14.el7

Comment 4 Sibiao Luo 2014-11-28 07:23:09 UTC
Verify this issue on qemu-kvm-rhev-2.1.2-14.el7.x86_64.

host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-205.el7.x86_64
qemu-kvm-rhev-2.1.2-14.el7.x86_64

Scenario 1:
-----------
# sync; echo 3 > /proc/sys/vm/drop_caches
# free -m;qemu-img convert -t none -f raw -O qcow2 image-1.raw image-test.qcow2; free -m
              total        used        free      shared  buff/cache   available
Mem:           7781         322        7293           9     ***165***     7266
Swap:          8063           0        8063
              total        used        free      shared  buff/cache   available
Mem:           7781         323        2414           9     ***5042***     7211
Swap:          8063           0        8063

# sync; echo 3 > /proc/sys/vm/drop_caches
[root@dhcp-11-154 home]# free -m;qemu-img convert -t none -f raw -O qcow2 image-1.raw image-test.qcow2 -T none; free -m
              total        used        free      shared  buff/cache   available
Mem:           7781         325        7291           9      ***164***     7263
Swap:          8063           0        8063
              total        used        free      shared  buff/cache   available
Mem:           7781         326        7272           9      ***183***     7254
Swap:          8063           0        8063

As we can see above, cached column differs a lot from without '-T none' and with '-T none'.

Scenario 2:
-----------
If you call "strace -e trace=open qemu-img xxx -T none source.img destination.img", the last time each file (source.img and destination.img) is opened with the ***O_DIRECT*** flag.

2.1 qemu-img convert.
# git diff convert-without-T.log convert-with-T.log 
diff --git a/convert-without-T.log b/convert-with-T.log
index 87da227..402095c 100644
--- a/convert-without-T.log
+++ b/convert-with-T.log
@@ -80,7 +80,7 @@ open("/proc/sys/crypto/fips_enabled", O_RDONLY) = 3
 open("/dev/urandom", O_RDONLY)          = 3
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
-open("image-1.raw", O_RDONLY|O_CLOEXEC) = 7
+open("image-1.raw", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8

2.2 qemu-img check.
# git diff check-without-T.log check-with-T.log 
diff --git a/check-without-T.log b/check-with-T.log
index 7f7602d..64d6b5a 100644
--- a/check-without-T.log
+++ b/check-with-T.log
@@ -80,5 +80,5 @@ open("/proc/sys/crypto/fips_enabled", O_RDONLY) = 3
 open("/dev/urandom", O_RDONLY)          = 3
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 7
+open("image-1.qcow2", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 7
 +++ exited with 0 +++

2.3 qemu-img compare.
# git diff compare-without-T.log compare-with-T.log 
diff --git a/compare-without-T.log b/compare-with-T.log
index 18efed1..814792c 100644
--- a/compare-without-T.log
+++ b/compare-with-T.log
@@ -80,8 +80,8 @@ open("/proc/sys/crypto/fips_enabled", O_RDONLY) = 3
 open("/dev/urandom", O_RDONLY)          = 3
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
-open("image-1.raw", O_RDONLY|O_CLOEXEC) = 7
+open("image-1.raw", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 8
+open("image-1.qcow2", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 8
 +++ exited with 0 +++

2.4 qemu-img rebase.
# git diff rebase-without-T.log rebase-with-T.log 
diff --git a/rebase-without-T.log b/rebase-with-T.log
index 28d697f..b847a66 100644
--- a/rebase-without-T.log
+++ b/rebase-with-T.log
@@ -86,8 +86,8 @@ open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 9
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 9
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 9
+open("image-1.qcow2", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 9
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 10
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 10
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 10
+open("image-1.qcow2", O_RDONLY|O_***DIRECT***|O_CLOEXEC) = 10
 +++ exited with 0 +++

2.5 qemu-img amend.
# git diff amend-without-T.log amend-with-T.log
diff --git a/amend-without-T.log b/amend-with-T.log
index 78d28cb..a507c5b 100644
--- a/amend-without-T.log
+++ b/amend-with-T.log
@@ -80,5 +80,5 @@ open("/proc/sys/crypto/fips_enabled", O_RDONLY) = 3
 open("/dev/urandom", O_RDONLY)          = 3
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
-open("image-1.qcow2", O_RDWR|O_CLOEXEC) = 7
+open("image-1.qcow2", O_RDWR|O_***DIRECT***|O_CLOEXEC) = 7
 +++ exited with 0 +++

Base on above, this issue has been fixed correctly, move to VERIFIED status, please correct me if any mistake, thanks.

Best Regards,
sluo

Comment 6 errata-xmlrpc 2015-03-05 09:58:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0624.html


Note You need to log in before you can comment on or make changes to this bug.