Bug 845702 - Regression (high system load)
Regression (high system load)
Product: Fedora
Classification: Fedora
Component: nginx (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Jeremy Hinegardner
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2012-08-03 18:48 EDT by Konstantin Svist
Modified: 2013-02-13 09:56 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-02-13 09:56:06 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Linux Kernel 45471 None None None 2012-08-03 18:48:06 EDT

  None (edit)
Description Konstantin Svist 2012-08-03 18:48:06 EDT
I have a bunch of servers with same hardware running different kernel revisions
(trying to upgrade them gradually). What I've noticed is that latest revisions
seem to have a much higher load average (see below).
To be fair, the first machine is Fedora 14, but the last two are running exact
same software (except for the kernel version) -- I don't think there should be
this much performance impact.

Linux hst624 #1 SMP Thu Sep 1 11:59:56 UTC 2011 x86_64
x86_64 x86_64 GNU/Linux

top - 17:56:58 up 32 days, 19:15,  1 user,  load average: 0.98, 0.95, 0.87
Tasks: 833 total,   7 running, 825 sleeping,   0 stopped,   1 zombie
Cpu(s): 32.3%us, 13.4%sy,  0.0%ni, 52.3%id,  0.0%wa,  0.0%hi,  2.0%si,  0.0%st
Mem:  132292864k total, 100893356k used, 31399508k free,   157460k buffers
Swap:  8191996k total,    11804k used,  8180192k free, 19326420k cached

Linux hst641 3.3.7-1.fc16.x86_64 #1 SMP Tue May 22 13:59:39 UTC 2012 x86_64
x86_64 x86_64 GNU/Linux

top - 17:57:38 up 15 min,  2 users,  load average: 5.30, 4.07, 2.76
Tasks: 569 total,   9 running, 560 sleeping,   0 stopped,   0 zombie
Cpu(s): 24.6%us, 10.0%sy,  0.0%ni, 61.8%id,  0.0%wa,  2.0%hi,  1.6%si,  0.0%st
Mem:  132032624k total, 78058084k used, 53974540k free,    69580k buffers
Swap:  8191996k total,        0k used,  8191996k free, 11310772k cached

Linux hst623 3.4.6-1.fc16.x86_64 #1 SMP Fri Jul 20 12:58:04 UTC 2012 x86_64
x86_64 x86_64 GNU/Linux

top - 17:58:05 up  2:17,  2 users,  load average: 24.82, 25.66, 26.34
Tasks: 576 total,   6 running, 570 sleeping,   0 stopped,   0 zombie
Cpu(s): 25.6%us, 10.4%sy,  0.0%ni, 60.2%id,  0.0%wa,  2.2%hi,  1.6%si,  0.0%st
Mem:  132032544k total, 91066796k used, 40965748k free,    84332k buffers
Swap:  8191996k total,        0k used,  8191996k free, 11397696k cached

Version 3.4.2-1 still has load ~5-6, so the latest regression is between that
and 3.4.6-1

Load from ~1 to ~6 seems to originate somewhere between
and -- can't find easily installable versions inbetween those
so far
Comment 1 Konstantin Svist 2012-08-03 18:49:46 EDT
Note: 2.6.38 and 2.6.43 tested on Fedora 14 system by installing these kernels manually. Updating to glibc-2.14 (from 2.13) doesn't seem to matter, since booting to 2.6.38 solves the problem
Comment 2 Konstantin Svist 2012-08-07 18:02:34 EDT
After testing many different kernel versions, I'm more confused than ever.

Same kernel version on F14 and F16 ( gives me verydifferent results. The main culprit seems to be nginx process, which (according to sysprof) calls __close_nocancel -> kernel. In F14, it takes up ~5%, but in F16 (same kernel!) it takes up ~57% (most of which seems to be used on __mutex_lock_common.clone.5 35% and do_raw_spin_lock 13%)

Load averages:
f14: 4.70, 5.60, 5.95
f16: 36.99, 36.37, 32.72

Can anyone help?
Comment 3 Justin M. Forbes 2012-08-24 15:43:16 EDT
Clearly not the kernel then. I would guess nginx itself or something it depends on.  Reassigning to them so they might figure out what the issue is.
Comment 4 Fedora End Of Life 2013-01-16 08:59:22 EST
This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
Comment 5 Fedora End Of Life 2013-02-13 09:56:09 EST
Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.