Bug 648657 - fseek()/NFS performance regression between RHEL4 and RHEL5
Summary: fseek()/NFS performance regression between RHEL4 and RHEL5
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel
Version: 5.7
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Jeff Layton
QA Contact: Jian Li
URL:
Whiteboard:
: 696207 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-11-01 21:13 UTC by Jeremy Eder
Modified: 2018-11-14 13:36 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-07-21 10:09:06 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
patch: allow redirtying of completed unstable write (2.26 KB, patch)
2010-11-05 14:45 UTC, Jeff Layton
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:1065 0 normal SHIPPED_LIVE Important: Red Hat Enterprise Linux 5.7 kernel security and bug fix update 2011-07-21 09:21:37 UTC

Comment 14 Jeff Layton 2010-11-03 15:42:22 UTC
We could, but I don't think it'll help, as Jeremy's testing of my kernels shows. The code will still require that the data be flushed to the server and committed. 

So instead of 20000 stable writes, you'll have 20000 unstable writes and ~20000 commits. Not exactly a win.

Comment 15 Jeff Layton 2010-11-03 15:44:15 UTC
What we really want to happen in this case is for the fstat call to cause dirty data to be written to the server but not necessarily committed. That won't make this as fast as RHEL4 (since it cheats), but it should put RHEL5 on par with RHEL6's performance.

Comment 16 Jeff Layton 2010-11-05 14:45:46 UTC
Created attachment 458112 [details]
patch: allow redirtying of completed unstable write

This patch seems to bring performance closer into line with RHEL6. The idea here is to eliminate the unnecessary COMMIT call when we're redirtying an already written page.

On my virtualized test rig:

$ /tmp/test
 beginning of the loop 
 diff is 4108 for 10000 loops

Call counts:

WRITE:  20000
COMMIT: 2

The patch is only lightly tested, but connectathon passes. It should apply cleanly on top of the jtltest.123 kernels on my PRC page:

    http://people.redhat.com/jlayton/

Comment 17 Jeff Layton 2010-11-05 14:53:37 UTC
For comparison, running on a jtltest.123 kernel:

$ /tmp/test
 beginning of the loop 
 diff is 256128 for 10000 loops

WRITE: 20000
COMMIT: 19988

...so this is a substantial improvement on this test.

Comment 18 Jeff Layton 2010-11-06 12:28:30 UTC
I've built some kernels with the above patch and put them on my people.redhat.com page. Please test them if you're able:

    http://people.redhat.com/jlayton/

Comment 23 RHEL Program Management 2011-02-01 16:52:33 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 27 Jarod Wilson 2011-02-21 20:56:55 UTC
in kernel-2.6.18-245.el5
You can download this test kernel (or newer) from http://people.redhat.com/jwilson/el5

Detailed testing feedback is always welcomed.

Comment 36 Jeff Layton 2011-04-14 17:57:29 UTC
*** Bug 696207 has been marked as a duplicate of this bug. ***

Comment 37 Robin R. Price II 2011-04-18 17:59:13 UTC
Any progress on our need findings from BZ 
https://bugzilla.redhat.com/show_bug.cgi?id=696207 ?

Thanks.

~rp

Comment 38 Robin R. Price II 2011-04-18 17:59:42 UTC
Any progress on our new findings from BZ 
https://bugzilla.redhat.com/show_bug.cgi?id=696207 ?

Thanks.

~rp

Comment 39 Jeff Layton 2011-04-18 18:29:03 UTC
What new findings? As best I can tell that bug turned out to be a duplicate of this one after all. The kernel they tested previously didn't have all of the patches necessary to resolve this. The most recent stock kernels do.

Comment 40 Robin R. Price II 2011-04-18 19:06:22 UTC
Ah.  I did not see that Kevin had tested with the latest stock.

"Your latest test kernel looks good.  As a sanity check, I ran updated my HS20
blade and test ppc64 hosts.  The test runs much better with the
2.6.18-254.el5.jtltest.137 kernel.  Thanks."

Thank you for pointing that out.

me--

~rp

Comment 41 Jian Li 2011-06-13 02:30:40 UTC
In this test, patch is checked, three test programs above are ran using 3 kernels(2.6.18-243.el5,2.6.18-262.el5,2.6.32-128.el6.x86_64), result is listed below, with commit 19, this bug can be verified.

[root@beaker rhel5]# git log 2.6.18-244.el5...2.6.18-245.el5 --oneline | grep "allow redirtying of a completed unstable write"
124d29d [fs] nfs: allow redirtying of a completed unstable write

2.6.18-243.el5
test
 beginning of the loop
 diff is 318543 for 10000 loops
test_fast
 beginning of the loop
 diff is 37 for 10000 loops
fseektest
write byte per sec: 2812
write byte per sec: 3040
write byte per sec: 2888
** ** snip ** **

2.6.18-262.el5
test
 beginning of the loop
 diff is 1001 for 10000 loops
test_fast
 beginning of the loop
 diff is 35 for 10000 loops
fseektest
write byte per sec: 569240
write byte per sec: 1611580
write byte per sec: 1620396
** ** snip ** **

2.6.32-128.el6.x86_64
test
 beginning of the loop
 diff is 1866 for 10000 loops
test_fast
 beginning of the loop
 diff is 31 for 10000 loops
fseektest
write byte per sec: 42940
write byte per sec: 954636
write byte per sec: 956612
** ** snip ** **

Comment 42 errata-xmlrpc 2011-07-21 10:09:06 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-1065.html


Note You need to log in before you can comment on or make changes to this bug.