This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours

Bug 865337

Summary: RFE: add aio=native (or threads) based on the storage where the images are stored
Product: Red Hat OpenStack Reporter: Yaniv Kaul <ykaul>
Component: openstack-novaAssignee: Daniel Berrange <berrange>
Status: ASSIGNED --- QA Contact: Shai Revivo <srevivo>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 4.0CC: apevec, berrange, bsettle, dmaley, famz, jdonohue, jeder, markmc, pbonzini, sclewis, scohen, sgordon
Target Milestone: ---Keywords: CodeChange, FutureFeature, Triaged, ZStream
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
URL: https://blueprints.launchpad.net/nova/+spec/improve-nova-kvm-io-support
Whiteboard: storage upstream_milestone_none upstream_status_unknown upstream_definition_drafting
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:

Description Yaniv Kaul 2012-10-11 05:12:08 EDT
Description of problem:
On file-based storage, aio=threads should be used, on block-based storage, aio=native should be used in the -drive command line of QEMU, for improved IO performance.

Version-Release number of selected component (if applicable):
Essex

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 3 Nikola Dipanov 2012-12-18 06:25:55 EST
This seems to be an issue of adding io=thread or io=native to the driver subnode of the disk node in libvirt xml when starting instances/attaching volumes. (also see https://bugzilla.redhat.com/show_bug.cgi?id=591703 for more background)

I will move this to 3.0 Grizzly release as does not seem to be a critical fix.

Also it would be good to have some more details on the actual benefits of fixing this upstream so adding a needinfo flag.
Comment 4 Daniel Berrange 2012-12-18 06:47:17 EST
We need the info on why each mode is better for block vs file, so we can figure out what Nova needs todo for network based storage. We also need to be able to backup our assertion that these modes are better with real facts when submitting the change upstream.
Comment 9 Daniel Berrange 2013-11-13 07:12:25 EST
*** Bug 1029596 has been marked as a duplicate of this bug. ***
Comment 13 Fam Zheng 2015-03-10 02:57:12 EDT
Cisco is seeing significant performance difference with aio=native compared to aio=thread on RHEL 7.

Fam
Comment 14 Stephen Gordon 2015-03-16 14:53:04 EDT
(In reply to Fam Zheng from comment #13)
> Cisco is seeing significant performance difference with aio=native compared
> to aio=thread on RHEL 7.
> 
> Fam

Hi Fam,

As Dan touches on in comment # 3 just saying "native" works better or "threads" works better isn't particularly useful in guiding us to a resolution without more information on whether it's file or block access, whether it's network storage, if so which protocol, and the measurable performance gap.

We know that native won't be better in all circumstances (and conversely that threads won't be better in all circumstances).

Thanks,

Steve
Comment 15 Fam Zheng 2015-03-16 21:01:13 EDT
This is local file based image. The details are in bug 1175440.

My impression is that native is likely better than threads whenever it is available, but unfortunately we default to threads. Is it even configurable in nova? How do we enable native?
Comment 16 Daniel Berrange 2015-03-17 06:08:41 EDT
(In reply to Fam Zheng from comment #15)
> This is local file based image. The details are in bug 1175440.
> 
> My impression is that native is likely better than threads whenever it is
> available, but unfortunately we default to threads. Is it even configurable
> in nova? How do we enable native?

We don't expose it as a configurable option to users because it is a low level hypervisor detail that is inappropriate to expose in a cloud model. We need to be able to have Nova set the appropriate setting automatically, but we cannot do that based on gut feeling / intuition. When we propose a particular setting upstream we need to have actual data to backup our assertion that it has better performance
Comment 17 Paolo Bonzini 2015-03-27 07:32:06 EDT
Comment 0 does not tell the whole story.  At least for ext4 io=threads is a huge loss (bug 1059744) and for XFS it is a moderate loss.  We need to test again NFS but I suspect that with recent-enough QEMU io=native is always a win.
Comment 18 Mark McLoughlin 2015-11-19 15:47:31 EST
Related upstream spec: https://review.openstack.org/#/c/232514/