Bug 656042
Summary: | [RFE] Include autogroup patch to aid in automatic creation of cgroups | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Jeremy West <jwest> |
Component: | kernel | Assignee: | Vivek Goyal <vgoyal> |
Status: | CLOSED ERRATA | QA Contact: | Jan Tluka <jtluka> |
Severity: | high | Docs Contact: | |
Priority: | high | ||
Version: | 6.1 | CC: | arozansk, mzywusko, snagar, ttracy |
Target Milestone: | rc | Keywords: | FutureFeature, Triaged |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | kernel-2.6.32-117.el6 | Doc Type: | Enhancement |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2011-05-23 20:29:33 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 662543 |
Description
Jeremy West
2010-11-22 22:17:44 UTC
This request was evaluated by Red Hat Product Management for inclusion in the current release of Red Hat Enterprise Linux. Because the affected component is not scheduled to be updated in the current release, Red Hat is unfortunately unable to address this request at this time. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux. If you would like it considered as an exception in the current release, please ask your support representative. This request was erroneously denied for the current release of Red Hat Enterprise Linux. The error has been fixed and this request has been re-proposed for the current release. This request was evaluated by Red Hat Product Management for inclusion in the current release of Red Hat Enterprise Linux. Because the affected component is not scheduled to be updated in the current release, Red Hat is unfortunately unable to address this request at this time. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux. If you would like it considered as an exception in the current release, please ask your support representative. This request was erroneously denied for the current release of Red Hat Enterprise Linux. The error has been fixed and this request has been re-proposed for the current release. Patch(es) available on kernel-2.6.32-112.el6 Built fresh RHEl6.1-20110207.3 systems on Intel Westmere. When I started testing, I noticed the performance had dropped significantly using the .113 kernel. So I went and tested the .94 kernel and the results were what I reported in January. Next I tested various kernels to pinpoint which kernel started this regression. Found that starting with the .112 kernel and further and regression is apparent. Below is an example MRG 1.3 with 2.6.32-111.el6.x86_64 kernel # of Pubs # of Subs Transfers/Sec MB/Sec 70354.1 67794.1 1.56402e+06 11.9325 69272.4 66420.6 1.55439e+06 11.859 79369.8 70189.8 1.08535e+06 8.28059 69252 66698.1 1.5352e+06 11.7127 68242.8 65861.3 1.53814e+06 11.7351 68892.4 66442.1 1.55164e+06 11.838 69170.6 66748.6 1.59185e+06 12.1448 68935.4 66392.8 1.54703e+06 11.8029 68947.6 66228.2 1.54834e+06 11.8129 69211.3 66919.8 1.5436e+06 11.7768 68625.4 65933.9 1.54062e+06 11.754 MRG1.3 with 2.6.32-112.el6.x86_64 kernel # of Pubs # of Subs Transfers/Sec MB/Sec 53283.1 51093 1.21434e+06 9.26468 51543 49941.3 1.18038e+06 9.00561 51819.4 50174.1 1.18401e+06 9.03328 51279.9 49856.3 1.17822e+06 8.98911 51442.6 49824.5 1.17183e+06 8.94037 51422.7 50125.5 1.18574e+06 9.04649 51865.6 50458.5 1.20652e+06 9.20499 51363.1 50099.6 1.17458e+06 8.96132 51314.9 49555.3 1.17232e+06 8.94411 51769.2 50297.4 1.18347e+06 9.02915 51551.5 49878.2 1.17719e+06 8.98127 51727.1 50104.6 1.18235e+06 9.02063 51378 49870.1 1.18098e+06 9.01018 As you can see I was averaging ~1.55 million Transfers/Sec at 11.5MB/Sec. Now with the 112 and higher kernels the throughput has dropped to 1.17 million Transfer/Sec at 9MB/Sec The changelog indicated on the 112 autogroup was installed and I tried disabling it to no avail. Here is what else has changed in the 112 kernel I have taken oprofile results running on the .111 and .112 kernels. There are also perf top logs for each kernel With the 111 kernel, here are the top routines samples % image name app name symbol name 4846706 28.8400 libqpidbroker.so.4.0.0 qpidd /usr/lib64/libqpidbroker.so.4.0.0 3976948 23.6645 libqpidcommon.so.4.0.0 qpidd /usr/lib64/libqpidcommon.so.4.0.0 641869 3.8194 libc-2.12.so qpidd _int_malloc 594161 3.5355 libstdc++.so.6.0.13 qpidd /usr/lib64/libstdc++.so.6.0.13 563315 3.3520 libpthread-2.12.so qpidd pthread_mutex_lock 488250 2.9053 vmlinux vmlinux futex_wake 453144 2.6964 libpthread-2.12.so qpidd pthread_mutex_unlock 419855 2.4983 libc-2.12.so qpidd _int_free 332677 1.9796 libc-2.12.so qpidd malloc 299754 1.7837 vmlinux vmlinux futex_wait_setup With the .112 kernel you see futex wait climb from 2.9% to 12.6% along with futex_wait_setup and tg_shares_up samples % image name app name symbol name 3982579 22.0255 libqpidbroker.so.4.0.0 qpidd /usr/lib64/libqpidbroker.so.4.0.0 3317650 18.3481 libqpidcommon.so.4.0.0 qpidd /usr/lib64/libqpidcommon.so.4.0.0 2295122 12.6931 vmlinux vmlinux futex_wake 1759122 9.7287 vmlinux vmlinux futex_wait_setup 833752 4.6110 vmlinux vmlinux tg_shares_up 525718 2.9075 libc-2.12.so qpidd _int_malloc 486086 2.6883 libstdc++.so.6.0.13 qpidd /usr/lib64/libstdc++.so.6.0.13 449337 2.4850 libpthread-2.12.so qpidd pthread_mutex_lock 368629 2.0387 libpthread-2.12.so qpidd pthread_mutex_unlock 290155 1.6047 libc-2.12.so qpidd _int_free 257811 1.4258 libc-2.12.so qpidd malloc Talking with Vivek, he built a test kernel with disables the autogroup unless it is requested. With the test kernel and autogroup disabled, performance returned to previous kernels. Oprofile results show similar characteristics of the 111 kernel. With kernel-2.6.32-115.el6.vgoyal.autogroupfix.v1.x86_64 testing Broadcom 1Gb network card Autogroup enabled in 112 kernel 8 Bytes Transfers/Sec MB/Sec 53283.1 51093 1.21434e+06 9.26468 51543 49941.3 1.18038e+06 9.00561 51819.4 50174.1 1.18401e+06 9.03328 51279.9 49856.3 1.17822e+06 8.98911 51442.6 49824.5 1.17183e+06 8.94037 51422.7 50125.5 1.18574e+06 9.04649 51865.6 50458.5 1.20652e+06 9.20499 Autogroup Diabled in kernel 8 Bytes MB/Sec (samall sample) 1.58E+006 12.07 1.60E+006 12.17 1.13E+006 8.63 1.59E+006 12.09 1.60E+006 12.22 1.59E+006 12.15 1.60E+006 12.21 1.58E+006 12.07 Patch(es) available on kernel-2.6.32-117.el6 An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0542.html |