Bug 963623 - The libvirt didn't bypass the the file system cache while set the auto_start_bypass_cache = 1 in qemu.conf
The libvirt didn't bypass the the file system cache while set the auto_start_...
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.0
x86_64 Linux
medium Severity medium
: rc
: ---
Assigned To: Eric Blake
Virtualization Bugs
:
Depends On:
Blocks: 963624
  Show dependency treegraph
 
Reported: 2013-05-16 05:29 EDT by zhenfeng wang
Modified: 2013-05-20 22:13 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 963624 (view as bug list)
Environment:
Last Closed: 2013-05-20 22:13:47 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description zhenfeng wang 2013-05-16 05:29:01 EDT
Description of problem:
The libvirt didn't bypass the the file system cache while restoring the guest from managed save file and set the auto_start_bypass_cache = 1 in qemu.conf

Version-Release number of selected component (if applicable):
libvirt-1.0.5-1.el7.x86_64
kernel-3.9.0-0.55.el7.x86_64
qemu-kvm-1.4.0-4.el7.x86_64
How reproducible:
100%

Steps
1.set the auto_start_bypass_cache = 1 in qemu.conf ,then restart libvirtd service
cat /etc/libvirt/qemu.conf
--
auto_start_bypass_cache = 1
--
2. start a guest and make sure the autostart is enabled
#virsh autostart rhelguest1

#virsh start rhelguest1

# virsh dominfo rhelguest1
Id:             1
Name:           rhelguest1
UUID:           fcf773e6-6eba-4e72-8299-de0a38c89136
OS Type:        hvm
State:          running
CPU(s):         1
CPU time:       16.9s
Max memory:     1048576 KiB
Used memory:    1048576 KiB
Persistent:     yes
Autostart:      enable

3.after the guest start completely,managedsave the guest
# virsh managedsave rhelguest1

4 restore the guest directly while the guest managedsave successfully
# virsh start rhel6  
Open another terminal, excute the following command before the guest finished starting,
found that the bypass-cache flag was not included in the following output

#while(true);do cat /proc/$(lsof -w /var/lib/libvirt/qemu/save/rhel6.save|awk '/libvirt_i/{print $2}')/fdinfo/*0*  ;done

this output didn't include the bypass-cache flag

5.however if start the domain with --bypass-cache ,we can get the expect value

# virsh start rhel6  --bypass-cache
Open another terminal, excute the following command before the guest finished starting,
you can see the third value was "4" in flags segment,it means --bypass-cache works

#while(true);do cat /proc/$(lsof -w /var/lib/libvirt/qemu/save/rhel6.save|awk '/libvirt_i/{print $2}')/fdinfo/*0*  ;done
pos:        34603008
flags:        0140000
pos:        163577856
flags:        0140000


Actual results:
In step 4 , we didn't found the bypass-cache flag in the second terminal , so the set about auto_start_bypass_cache=1 in qemu.conf didn't work

Expected results:
As the the auto_start_bypass_cache description in qemu.conf,When a domain is configured to be auto-started, enabling this flag should
has the same effect as using the VIR_DOMAIN_START_BYPASS_CACHE flag with the virDomainCreateWithFlags API.  That is, the system will
avoid using the file system cache when restoring any managed state file however. in my test they are different
Comment 2 Eric Blake 2013-05-16 10:33:33 EDT
(In reply to comment #0)
> Description of problem:
> The libvirt didn't bypass the the file system cache while restoring the
> guest from managed save file and set the auto_start_bypass_cache = 1 in
> qemu.conf

auto_start_bypass_cache ONLY affects whether auto-starting a domain will bypass cache.  But you polluted the cache when you did...

> 3.after the guest start completely,managedsave the guest
> # virsh managedsave rhelguest1

...managedsave without --bypass-cache.  Then you also polluted the cache...


> 
> 4 restore the guest directly while the guest managedsave successfully
> # virsh start rhel6  

...when you did a manual start instead of an autostart, and again omitted --bypass-cache.

For this test to work correctly, you must use the --bypass-cache option to both virsh commands.

I could see this being turned into an RFE to make libvirt behave as if --bypass-cache were always present on ALL save/start operations, without you having to type it on the virsh command line; but my worry is that --bypass-cache fails for images on tmpfs (or any other file system that lacks O_DIRECT support), so globally enabling it may cause problems to existing clients.  The current qemu.conf setting is an opt-in only for the autostart sequence, since that is the only sequence where you don't have explicit control over whether to use the flag.

> 
> Expected results:
> As the the auto_start_bypass_cache description in qemu.conf,When a domain is
> configured to be auto-started, enabling this flag should
> has the same effect as using the VIR_DOMAIN_START_BYPASS_CACHE flag with the
> virDomainCreateWithFlags API.  That is, the system will
> avoid using the file system cache when restoring any managed state file
> however. in my test they are different

You didn't test autostart.  You need to fix your test to observe autostart behavior, rather than 'virsh start' behavior.

I'll wait for feedback from others on whether we should close this as NOTABUG, or whether we should convert it into an RFE to make the qemu.conf setting control more than just autostart.
Comment 3 Eric Blake 2013-05-16 10:35:01 EDT
For the record, the way to test autostart is to stop libvirtd then restart it.  If you have to type 'virsh start', then you aren't testing autostart.
Comment 4 Daniel Berrange 2013-05-20 06:29:19 EDT
IMHO this is NOTABUG, the docs for the 'auto_start_bypass_cache' parameter are pretty clear about what it does. As you say, turning this on globally for more opens up a can of worms due to lack of O_DIRECT support in a number of cases.
Comment 5 zhenfeng wang 2013-05-20 22:10:35 EDT
I' sorry to misundterstand the manpage, I can get the expect result while i try to start the guest with libvirtd service

1.set the auto_start_bypass_cache = 1 in qemu.conf ,then restart libvirtd service
cat /etc/libvirt/qemu.conf
--
auto_start_bypass_cache = 1
--
2. start a guest and make sure the autostart is enabled
#virsh autostart rhelguest1

#virsh start rhelguest1

# virsh dominfo rhelguest1
Id:             1
Name:           rhelguest1
UUID:           fcf773e6-6eba-4e72-8299-de0a38c89136
OS Type:        hvm
State:          running
CPU(s):         1
CPU time:       16.9s
Max memory:     1048576 KiB
Used memory:    1048576 KiB
Persistent:     yes
Autostart:      enable

3.after the guest start completely,managedsave the guest
# virsh managedsave rhelguest1
4 Stop the libvirtd service while the guest managedsave successfully
#service libvirtd stop
5 start the libvirtd service ,at the same time ,open another terminal, excute the following command before the guest finished starting,
you can see the third value was "4" in flags segment,it means --bypass-cache works

#while(true);do cat /proc/$(lsof -w /var/lib/libvirt/qemu/save/rhelguest1.save|awk '/libvirt_i/{print $2}')/fdinfo/*0*  ;done
pos:        34603008
flags:        0140000
pos:        163577856
flags:        0140000
Comment 6 Eric Blake 2013-05-20 22:13:47 EDT
Closing this as NOTABUG.

Note You need to log in before you can comment on or make changes to this bug.