Bug 626963 - AIO uses igrab in the submission path, which causes undue lock contention
Summary: AIO uses igrab in the submission path, which causes undue lock contention
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel
Version: 5.6
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Jeff Moyer
QA Contact: Kernel Performance QE
URL:
Whiteboard:
Depends On: 626595
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-08-24 18:41 UTC by Jeff Moyer
Modified: 2011-01-13 21:12 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 626595
Environment:
Last Closed: 2011-01-13 21:12:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
change igrab to atomic_inc (886 bytes, patch)
2010-08-24 18:44 UTC, Jeff Moyer
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:0017 0 normal SHIPPED_LIVE Important: Red Hat Enterprise Linux 5.6 kernel security and bug fix update 2011-01-13 10:37:42 UTC

Description Jeff Moyer 2010-08-24 18:41:59 UTC
+++ This bug was initially created as a clone of Bug #626595 +++

Description of problem:
From Chris Mason:

The aio batching code is using igrab to get an extra reference on the
inode so it can safely batch.  igrab will go ahead and take the global
inode spinlock, which can be a bottleneck on large machines doing lots
of AIO.

In this case, igrab isn't required because we already have a reference
on the file handle.  It is safe to just bump the i_count directly
on the inode.

Benchmarking shows this patch brings IOP/s on tons of flash up by about
2.5X.


Version-Release number of selected component (if applicable):
2.6.32-68.el6

How reproducible:
100%

Steps to Reproduce:
1. Run TPC-C on a huge machine
2. run perf report -g and witness time spent on the inode_lock
3.

Comment 1 Jeff Moyer 2010-08-24 18:44:07 UTC
Created attachment 440727 [details]
change igrab to atomic_inc

Comment 2 RHEL Program Management 2010-08-24 18:49:47 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 5 Jarod Wilson 2010-09-15 14:01:38 UTC
in kernel-2.6.18-221.el5
You can download this test kernel from http://people.redhat.com/jwilson/el5

Detailed testing feedback is always welcomed.

Comment 8 errata-xmlrpc 2011-01-13 21:12:59 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0017.html


Note You need to log in before you can comment on or make changes to this bug.