Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 856144

Summary: Very long stalls when using commands which needs user information (e.g. w, ps axu)
Product: Red Hat Enterprise Linux 6 Reporter: Tommi Tervo <tommi.tervo>
Component: kernelAssignee: Larry Woodman <lwoodman>
kernel sub component: Memory Management QA Contact: Wang Shu <shuwang>
Status: CLOSED WORKSFORME Docs Contact:
Severity: medium    
Priority: unspecified CC: agk, aquini, cye, marco.passerini, wgomerin
Version: 6.3   
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-02 15:06:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1270638, 1359574, 1366045    
Attachments:
Description Flags
sysrq-t dumps
none
zoneinfo fetched during the stall
none
zoneinfo when khugepaged is taking 100% cputime
none
Strace of various commands which hang
none
perf top view none

Description Tommi Tervo 2012-09-11 10:02:03 UTC
Created attachment 611716 [details]
sysrq-t dumps

Description of problem:
w or ps axuwww can take 40 minutes sometimes to complete. 

Version-Release number of selected component (if applicable):
2.6.32-279.2.1.el6.x86_64

How reproducible:
Happens quite often but I don't know which type of load triggers this issue. Servers are multipurpose application servers. Sometimes khugepaged takes 100% cpu time but not always when this is happening. enabling/disabling zone_reclaim_mode doesn't make any difference.


Additional info:
HW: HP DL580 G7, 1TB ram, 4x Xeon X7560 8 core (total 32 cores, HT disabled)

Comment 1 Tommi Tervo 2012-09-11 10:03:31 UTC
Created attachment 611718 [details]
zoneinfo fetched during the stall

Comment 2 Tommi Tervo 2012-09-11 10:04:13 UTC
Created attachment 611719 [details]
zoneinfo when khugepaged is taking 100% cputime

Comment 4 RHEL Program Management 2012-12-14 08:25:13 UTC
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 5 Marco Passerini 2013-04-08 13:40:55 UTC
The problem persists, on the same server.
We are currently running the following OS version:

Red Hat Enterprise Linux Server release 6.4 (Santiago)
2.6.32-358.0.1.el6.x86_64

The "ps -ef" command stalls in the middle of execution, and usually it continues (and completes) after a couple of minutes of no activity. The same applies to commands such as "valgrind --help" , and "java -version".

Comment 6 Marco Passerini 2013-04-08 13:52:46 UTC
Created attachment 732679 [details]
Strace of various commands which hang

Comment 7 Tommi Tervo 2013-04-17 12:03:52 UTC
Created attachment 736818 [details]
perf top view

Culprit is THP, disabling it will cure this problem in couple of seconds.

When enabled, spin_lock_irqsave is on top of perf
15.79%  [kernel]  [k] _spin_lock_irqsave