Bug 601388

Summary: Performance degradation on newer kernel 2.6.18-194
Product: Red Hat Enterprise Linux 5 Reporter: infratirio
Component: kernelAssignee: Red Hat Kernel Manager <kernel-mgr>
Status: CLOSED NOTABUG QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: low    
Version: 5.5CC: cww, leotavaneiro, mgoodwin, rdassen
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-18 17:19:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description infratirio 2010-06-07 20:37:08 UTC
Description of problem:

     I'm using RHEL 5 on a cluster environment for seismic simulation. In this enviroment we are using 32 HP's DL160 G6 powered with two Intel X5570 CPU using the new nehalem microarchitecture. In the default DL160 configuration, the OS sees 16 processors instead of 8 because of the hyperthreading

We saw a performance degradation when we moved from RHEL 5.4 to RHEL 5.5. Our simulations took 10% more time to run on the newer RHEL 5.5 using the same simulation software and dataset.

Version-Release number of selected component (if applicable):

kernel kernel-2.6.18-194.el5 and kernel-2.6.18-194.3.1.el5

How reproducible:

    Just start a simulation job with 8 parallel process and wait for simulation timing results gave after the simulation job finished. In RHEL 5.4's kernel version kernel-2.6.18-164.el5 we can see a steady result of ~856 secods with less then 1 percent of standard deviation. This RHEL 5.5's kernel version kernel-2.6.18-194.el5 or even with the newest errata kernel-2.6.18-194.3.1.el5 we can experiment a 10% increase in the processing time with more than 30% of standard deviation.

Actual results:

This are the times we can get with kernel-2.6.18-194.el5 and kernel-2.6.18-194.3.1.el5

 Final cpu     987.82 elapsed     990.14
 Final cpu     903.95 elapsed     906.42
 Final cpu     999.24 elapsed    1001.67
 Final cpu     898.51 elapsed     900.83
 Final cpu     927.19 elapsed     929.51

Expected results:

This is the expected time and what we can get from kernel-2.6.18-164.el5

 Final cpu     857.43 elapsed     859.96
 Final cpu     856.02 elapsed     859.00
 Final cpu     855.96 elapsed     858.78
 Final cpu     856.09 elapsed     858.88
 Final cpu     855.83 elapsed     858.81

Additional info:

We did an aditional test that involved installing RHEL 5.5 and downgrading just the kernel to kernel-2.6.18-164.el5 and the results are the same as running a full install RHEL 5.4, which means the expected result time of ~856 seconds.

Comment 2 Chris Williams 2012-01-18 17:19:59 UTC
It has been some time since this BZ was updated. Both customer issues have closed due to lack of updates. Closing NOTABUG.
If this is still an issue, please feel free to open a case with Red Hat Support via the Customer Portal.