Bug 626963

Summary: AIO uses igrab in the submission path, which causes undue lock contention
Product: Red Hat Enterprise Linux 5 Reporter: Jeff Moyer <jmoyer>
Component: kernelAssignee: Jeff Moyer <jmoyer>
Status: CLOSED ERRATA QA Contact: Kernel Performance QE <kernel-perf-qe>
Severity: medium Docs Contact:
Priority: low    
Version: 5.6CC: arozansk, kkolakow, rwheeler, syeghiay
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 626595 Environment:
Last Closed: 2011-01-13 21:12:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 626595    
Bug Blocks:    
Attachments:
Description Flags
change igrab to atomic_inc none

Description Jeff Moyer 2010-08-24 18:41:59 UTC 
+++ This bug was initially created as a clone of Bug #626595 +++

Description of problem:
From Chris Mason:

The aio batching code is using igrab to get an extra reference on the
inode so it can safely batch.  igrab will go ahead and take the global
inode spinlock, which can be a bottleneck on large machines doing lots
of AIO.

In this case, igrab isn't required because we already have a reference
on the file handle.  It is safe to just bump the i_count directly
on the inode.

Benchmarking shows this patch brings IOP/s on tons of flash up by about
2.5X.


Version-Release number of selected component (if applicable):
2.6.32-68.el6

How reproducible:
100%

Steps to Reproduce:
1. Run TPC-C on a huge machine
2. run perf report -g and witness time spent on the inode_lock
3.
Comment 1 Jeff Moyer 2010-08-24 18:44:07 UTC 
Created attachment 440727 [details]
change igrab to atomic_inc
Comment 2 RHEL Program Management 2010-08-24 18:49:47 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 5 Jarod Wilson 2010-09-15 14:01:38 UTC 
in kernel-2.6.18-221.el5
You can download this test kernel from http://people.redhat.com/jwilson/el5

Detailed testing feedback is always welcomed.
Comment 8 errata-xmlrpc 2011-01-13 21:12:59 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0017.html