Bug 869347 - taskomatic fails during Errata import (ErrataMailer)
taskomatic fails during Errata import (ErrataMailer)
Status: NEW
Product: Spacewalk
Classification: Community
Component: Server (Show other bugs)
1.8
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Tomas Lestach
Red Hat Satellite QA List
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-23 12:03 EDT by JDavis4102
Modified: 2015-10-09 04:42 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description JDavis4102 2012-10-23 12:03:32 EDT
Description of problem: When importing Errata the taskomatic service fails and stops processing (with no error) during the erratamailer task.


Version-Release number of selected component (if applicable): 1.6 with PostgreSQL 9.1


How reproducible: Send a whole bunch of errata to Spacewalk. We have about 7000 machines and about 50 users in Spacewalk currently.

  
Actual results: Taskomatic fails without error and stops processing.


Expected results: Send email notifications and continue processing.

Additional info: We have disabled Errata-queue-bunch thinking this is the task that disables the errata mailer and it seems it was still generating email. Then we disabled the other errata tasks and found that for some reason the errata-cache-bunch is generating the emails. Without the errata-cache-bunch it seems that errata are of no use to us. 

I did fine a bug report that may explain the issue but I am unable to see the bug. Below is what I found that I think is our issue.

Due to the ErrataMailer task being too slow, processing a large number of
errata may have caused a running ErrataMailer task to keep blocking next tasks
until all available resources were exhausted and the Taskomatic job scheduler
stopped working. When this happened, no tasks (including the repository
generation task) were executed. With this update, the underlying source code of
the ErrataMailer task has been rewritten so that it no longer prevents
Taskomatic from working. (BZ#711194)

If this bug has already been fixed in a later release is there anyway that we could patch 1.6 with the fix or do we need to upgrade Spacewalk in order to fix this issue?
Comment 1 JDavis4102 2012-11-19 15:39:58 EST
Any updates as to this issue?
Comment 2 Jan Pazdziora 2013-03-08 10:27:18 EST
(In reply to comment #0)

> I did fine a bug report that may explain the issue but I am unable to see
> the bug. Below is what I found that I think is our issue.
> 
> Due to the ErrataMailer task being too slow, processing a large number of
> errata may have caused a running ErrataMailer task to keep blocking next
> tasks
> until all available resources were exhausted and the Taskomatic job scheduler
> stopped working. When this happened, no tasks (including the repository
> generation task) were executed. With this update, the underlying source code
> of
> the ErrataMailer task has been rewritten so that it no longer prevents
> Taskomatic from working. (BZ#711194)
> 
> If this bug has already been fixed in a later release is there anyway that
> we could patch 1.6 with the fix or do we need to upgrade Spacewalk in order
> to fix this issue?

That bugzilla was fixed in Spacewalk 1.2 so if you see a problem in Spacewalk 1.6, it's a different issue.
Comment 3 Tomas Lestach 2013-03-08 10:34:33 EST
I definitely wouldn't disable the taskomatic bunches you write about.
If you want to disable errata e-mail notifications, why don't you do it on the /rhn/account/UserPreferences.do page?
Comment 4 JDavis4102 2013-08-16 01:37:53 EDT
Sorry for the delay in reply. We have disabled the email notifications and that seems to have helped. Though it seems the Errata Cache runs all the time. In the Database I see a whole bunch of select distinct SQL queries regarding Errata. This increases load on the database server and slows down Spacewalk. This seems related to this issue as Errata processing is slowing the server down. We now have Spacewalk 1.8 installed and about 8400 servers using 21 different proxy servers. If you would like me to open a new ticket please let me know.
Comment 5 Tomas Lestach 2013-08-16 02:24:57 EDT
Just curious - what is the hardware you use for the server? I suppose you still use PostgreSQL as database backend. Does it run on a separate machine or on the same as Spacewalk?
Comment 6 JDavis4102 2013-08-26 13:26:06 EDT
The App server is a Dell PE R610 with 96G of RAM with Quad Core Xeon @ 1.87GHz.

Our PostgreSQL DB is on a separate machine that is a Dell PE R710 with 144G of RAM with a 2.27GHz Xeon.

Note You need to log in before you can comment on or make changes to this bug.