RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1138691 - Allow qemu-img to bypass the host cache (check, compare, convert, rebase, amend)
Summary: Allow qemu-img to bypass the host cache (check, compare, convert, rebase, amend)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Hanna Czenczek
QA Contact: Virtualization Bugs
URL:
Whiteboard: storage
Depends On: 1116558
Blocks: 1138690 1166481
TreeView+ depends on / blocked
 
Reported: 2014-09-05 13:08 UTC by Ademar Reis
Modified: 2015-03-05 08:11 UTC (History)
27 users (show)

Fixed In Version: qemu-kvm-1.5.3-79.el7
Doc Type: Rebase: Bug Fixes and Enhancements
Doc Text:
Clone Of: 1116558
: 1166481 (view as bug list)
Environment:
Last Closed: 2015-03-05 08:11:45 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0349 0 normal SHIPPED_LIVE Important: qemu-kvm security, bug fix, and enhancement update 2015-03-05 12:27:34 UTC

Comment 7 Miroslav Rezanina 2014-11-12 11:26:38 UTC 
Fix included in qemu-kvm-1.5.3-79.el7
Comment 9 Sibiao Luo 2014-11-21 05:07:12 UTC 
Verify this issue with the same steps as Chao Yang.

#######qemu-kvm-1.5.3-79.el7.x86_64:
host info:
# uname -r && rpm -q qemu-kvm
3.10.0-205.el7.x86_64
qemu-kvm-1.5.3-79.el7.x86_64

Scenario 1:
-----------
# sync; echo 3 > /proc/sys/vm/drop_caches
# free -m;qemu-img convert -t none -f raw -O qcow2 image-1.raw image-test.qcow2; free -m
              total        used        free      shared  buff/cache   available
Mem:           7781         290        7367           8     ***124***     7318
Swap:          8063          73        7990
              total        used        free      shared  buff/cache   available
Mem:           7781         290        2509           8     ***4981***     7253
Swap:          8063          73        7990

# sync; echo 3 > /proc/sys/vm/drop_caches
# free -m;qemu-img convert -t none -f raw -O qcow2 image-1.raw image-test.qcow2 -T none; free -m
              total        used        free      shared  buff/cache   available
Mem:           7781         290        7367           8      ***124***     7318
Swap:          8063          73        7990
              total        used        free      shared  buff/cache   available
Mem:           7781         291        7362           8      ***128***     7316
Swap:          8063          73        7990

As we can see above, cached column differs a lot from without '-T none' and with '-T none'.

Scenario 2:
-----------
If you call "strace -e trace=open qemu-img xxx -T none source.img destination.img", the last time each file (source.img and destination.img) is opened with the ***O_DIRECT*** flag.

2.1 qemu-img convert.
# git diff convert-without-T.log convert-with-T.log 
diff --git a/convert-without-T.log b/convert-with-T.log
index 5475bc7..dc75a8b 100644
--- a/convert-without-T.log
+++ b/convert-with-T.log
@@ -19,7 +19,7 @@ open("/lib64/libcrypto.so.10", O_RDONLY|O_CLOEXEC) = 3
 open("/lib64/libgpg-error.so.0", O_RDONLY|O_CLOEXEC) = 3
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 6
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 6
-open("image-1.raw", O_RDONLY|O_CLOEXEC) = 6
+open("image-1.raw", O_RDONLY|***O_DIRECT***|O_CLOEXEC) = 6
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7

2.2 qemu-img check.
# git diff check-without-T.log check-with-T.log 
diff --git a/check-without-T.log b/check-with-T.log
index d9a737a..b1e5f18 100644
--- a/check-without-T.log
+++ b/check-with-T.log
@@ -19,5 +19,5 @@ open("/lib64/libcrypto.so.10", O_RDONLY|O_CLOEXEC) = 3
 open("/lib64/libgpg-error.so.0", O_RDONLY|O_CLOEXEC) = 3
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 6
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 6
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 6
+open("image-1.qcow2", O_RDONLY|***O_DIRECT***|O_CLOEXEC) = 6
 +++ exited with 0 +++

2.3 qemu-img compare.
# git diff compare-without-T.log compare-with-T.log 
diff --git a/compare-without-T.log b/compare-with-T.log
index 646175e..afc45cc 100644
--- a/compare-without-T.log
+++ b/compare-with-T.log
@@ -19,8 +19,8 @@ open("/lib64/libcrypto.so.10", O_RDONLY|O_CLOEXEC) = 3
 open("/lib64/libgpg-error.so.0", O_RDONLY|O_CLOEXEC) = 3
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 6
 open("image-1.raw", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 6
-open("image-1.raw", O_RDONLY|O_CLOEXEC) = 6
+open("image-1.raw", O_RDONLY|***O_DIRECT***|O_CLOEXEC) = 6
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 7
+open("image-1.qcow2", O_RDONLY|***O_DIRECT***|O_CLOEXEC) = 7
 +++ exited with 0 +++

2.4 qemu-img rebase.
# git diff rebase-without-T.log rebase-with-T.log 
diff --git a/rebase-without-T.log b/rebase-with-T.log
index 451ae2c..0fbd0ec 100644
--- a/rebase-without-T.log
+++ b/rebase-with-T.log
@@ -25,8 +25,8 @@ open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_DIRECT|O_CLOEXEC) = 8
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 9
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 9
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 9
+open("image-1.qcow2", O_RDONLY|***O_DIRECT***|O_CLOEXEC) = 9
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 10
 open("image-1.qcow2", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 10
-open("image-1.qcow2", O_RDONLY|O_CLOEXEC) = 10
+open("image-1.qcow2", O_RDONLY|***O_DIRECT***|O_CLOEXEC) = 10
 +++ exited with 0 +++

2.5 qemu-img amend.
qemu-img amend fail to specify the cache mode to bypass the host cache in qemu-kvm-1.5.3-x verison(separated it to new bug 1166476 to trace it).


#######qemu-kvm-rhev-2.1.2-8.el7.x86_64:
host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-205.el7.x86_64
qemu-kvm-rhev-2.1.2-8.el7.x86_64

Scenario 1:
-----------
# sync; echo 3 > /proc/sys/vm/drop_caches
# free -m;qemu-img convert -t none -f raw -O qcow2 image-1.raw image-test.qcow2; free -m

# sync; echo 3 > /proc/sys/vm/drop_caches
# free -m;qemu-img convert -t none -f raw -O qcow2 image-1.raw image-test.qcow2 -T none; free -m

Scenario 2:
-----------
2.1 qemu-img convert.
2.2 qemu-img check.
2.3 qemu-img compare.
2.4 qemu-img rebase.
2.5 qemu-img amend.

It fail to work with qemu-kvm-rhev-2.1.2-X version which i have separated it to new bug 1166481 to trace it.

Bug 1166481 - Allow qemu-img to bypass the host cache (check, compare, convert, rebase, amend)

Base on above, this issue has been fixed, new issues have been separated to new bug to trace it. Move this issue to VERIFIED status first, thanks.

Best Regards,
sluo
Comment 12 errata-xmlrpc 2015-03-05 08:11:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0349.html
Note You need to log in before you can comment on or make changes to this bug.