Red Hat Bugzilla – Bug 681006
Abnormally high cpu usage in taskomatic after upgrade to Satellite 5.4
Last modified: 2011-05-23 15:07:09 EDT
Description of problem:
Customer repeatedly experiences very high cpu usage since upgrading to Satellite 5.4 stemming from taskomatic and more notably, it's java process.
CPU usage has not gone down in 3 weeks of running Satellite 5.4
Version-Release number of selected component (if applicable):
Satellite 5.4 -- latest packages.
100% at customer's site -- 3 to 4 other customers with this or similar problem. Not 100% sure yet if this is the same problem as those other customers -- working with fellow TAMs to figure that one out.
Steps to Reproduce:
1. Upgrade to Satellite 5.4
2. Be running quartz and quartz-oracle at the 1.8.1 level
Unexplained CPU hike that won't die down.
Normal CPU usage across the board.
In this particular case, we started by gathering thread dumps out of the taskomatic java process and noticed that the CPU usage time was being dominated by the Quartz Scheduler around lines 280-300.
Upon researching information on the Quartz Scheduler online, I found this bug:
The regression referred to by this bug is a 100% cpu utilization in quartz problem that hits right at line 287 -- the very part of the Quartz Scheduler that is acting up on the customer's setup.
This bug was introduced in 1.8.1 and was fixed in 1.8.3. The latest version of quartz is 1.8.4 and so I rolled unsupported rpms for quartz and quartz-oracle of 1.8.4 based on the spec found in brew. Those packages can be found at:
My customer was willing to test these packages even with the understanding that they were unsupported, testing packages. After upgrading to quartz-1.8.4, the abnormally high cpu usage went away immediately.
I would like to ask that we rebase quartz to 1.8.4 as soon as possible as I would like to move my customer to a supported configuration that doesn't have the abnormally high cpu usage problem.
Rebased to quartz-1.8.4, tagged and built.
This bug may impact rhn-search too as it uses the same quartz scheduler as taskomatic.
Old package(s) (quartz-1.8.1-3.el5sat):
Observed regular 100% CPU usage peaks caused by taskomatic Java processes.
New package(s) (quartz-1.8.4-1.el5sat):
The problem has gone, no unusual high CPU usage experienced.
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.