Bug 840862

Summary: kworker eating up a lot of cpu
Product: [Fedora] Fedora Reporter: Martijn Faassen <faassen>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 16CC: dan.waddell, gansalmon, itamar, jforbes, jonathan, kernel-maint, madhu.chinakonda
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-11-14 16:22:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martijn Faassen 2012-07-17 12:27:52 UTC
Description of problem:

I noticed a kworker process is eating about 65% of CPU.

Version-Release number of selected component (if applicable):

Fedora 16 Kernel 3.4.4-4.fc16.x86_64 (I tried downgrading to 2 earlier kernels but no luck)

How reproducible:

Boot up. Start 'top', and see kworker on top. I'm using lxde on this laptop, but I also see this before logging in.


Additional info:

I looked at the older bug 638912, and figured I might want to include this info:

Output of ls -ld /sys/bus/pci/slots/1/module

/sys/bus/pci/slots/1/module -> ../../../../module/acpiphp

watch -n 0.1 cat /sys/firmware/acpi/interrupts/gpe_all

produces a rapidly increasing count.

I also see a bug 829082 which reports this issue with Fedora 17.

Comment 1 Dan Waddell 2012-07-17 14:33:43 UTC
Perhaps this will help.  At least it's worth a shot. This is from the 3.4 kernel Documentation file Documentation/workqueue.txt:

----------------------------------------------------------------------
If kworkers are going crazy (using too much cpu), there are two types
of possible problems:

	1. Something beeing scheduled in rapid succession
	2. A single work item that consumes lots of cpu cycles

The first one can be tracked using tracing:

	$ echo workqueue:workqueue_queue_work > /sys/kernel/debug/tracing/set_event
	$ cat /sys/kernel/debug/tracing/trace_pipe > out.txt
	(wait a few secs)
	^C

If something is busy looping on work queueing, it would be dominating
the output and the offender can be determined with the work item
function.

For the second type of problems it should be possible to just check
the stack trace of the offending worker thread.

	$ cat /proc/THE_OFFENDING_KWORKER/stack

The work item's function should be trivially visible in the stack
trace.
----------------------------------------------------------------------

Comment 2 Martijn Faassen 2012-07-20 08:23:03 UTC
Thanks Dan! 

As to the first, I see a mix of things, at least I think so; quite a bit of idle, lxpanel, firefox, bash, etc.

A cat /proc/2/stack (as it's kworker/0:2 that's doing this, I assume I need 2), shows me only this:

[<ffffffff8107a095>] kthreadd+0x155/0x160
[<ffffffff816056e4>] kernel_thread_helper+0x4/0x10
[<ffffffffffffffff>] 0xffffffffffffffff

Comment 3 Dan Waddell 2012-07-20 13:39:53 UTC
It makes sense that you'll see lines for any and all processes that are running.  What I'd do is shutdown everything that you don't need running.  Check that you still see the issue then do these steps again.

Maybe even boot into single user mode just to quite the system as much as possible.  Check if the problem exists there as well.  Then execute these steps.

As far as the stack trace goes... thats a bit beyond me.  Perhaps someone else could comment on that.

Also, it doesn't usually hurt to do a "yum update" on everything.  Perhaps there is a patch for either the offend process or the kernel.

Comment 4 Josh Boyer 2012-09-05 16:38:32 UTC
Are you still having issues with this on 3.4.9 or newer?

Comment 5 Dave Jones 2012-10-23 15:37:23 UTC
# Mass update to all open bugs.

Kernel 3.6.2-1.fc16 has just been pushed to updates.
This update is a significant rebase from the previous version.

Please retest with this kernel, and let us know if your problem has been fixed.

In the event that you have upgraded to a newer release and the bug you reported
is still present, please change the version field to the newest release you have
encountered the issue with.  Before doing so, please ensure you are testing the
latest kernel update in that release and attach any new and relevant information
you may have gathered.

If you are not the original bug reporter and you still experience this bug,
please file a new report, as it is possible that you may be seeing a
different problem. 
(Please don't clone this bug, a fresh bug referencing this bug in the comment is sufficient).

Comment 6 Justin M. Forbes 2012-11-14 16:22:25 UTC
With no response, we are closing this bug under the assumption that it is no longer an issue. If you still experience this bug, please feel free to reopen the bug report.