+++ This bug was initially created as a clone of Bug #232902 +++
Description of problem:
Taken from a rhn-list posting and my own review. It seems that Auto Apply Errata
is not working as expected within RHN 5.0.0 since its release into production on
I do not have full details yet, but sounds like it kicked in for this account
and scheduled the application of errata to *all* systems on the account, even
those where the errata was not appliciable, and also ignoring the flag for 'Auto
Creating this *public* bug for tracking.
Version-Release number of selected component (if applicable):
Not sure yet
Report came from:
Steps to Reproduce:
I'm not sure whether this is a hosted or proxy issue and I'm not sure
if it is operator error or something else that caused it. We have a
hosted environment running a current release proxy server with custom
channels. A product enhancement errata was released for some of those
custom channels a couple of days ago and today it was scheduled for
every system subscribed to the pertinent channels (or so it seems at a
Question 1: Should this errata be scheduled for machines which don't
have any relevant package installed? It was but this seems silly. They
picked it up and failed to install it harmlessly.
Question 2: Should it be scheduled for systems with auto errata update
set to no? It was. I can't tell how the scheduling is done, it is
listed as (none).
Question 3: Should systems with this errata scheduled but which have
auto errata updates set to no pick up the package and install it when
doing the next rhn_check? They did. This caused major grief as it
broke openAFS in this case and it seems to me that this should not
Can anyone shed any light on any of this for me? It has been an
immensely long week and I'm exhausted so maybe I'm not thinking too
clearly at the moment.
-- Additional comment from email@example.com on 2007-03-19 12:31 EST --
Having reviewed, it seems that there is a bug in the background daemon that
schedules to application of 'Apply Auto Errata'. I have placed this bug onto the
must fix list for the next scheduled minor maintence release of the RHN Hosted
web site to get fixed.
-- Additional comment from firstname.lastname@example.org on 2007-03-20 15:42 EST --
* Fixed overly "generous" logic in the ErrataQueueWorker which was causing
auto-errata updates to be applied to too many machines.
-- Additional comment from email@example.com on 2007-03-22 10:09 EST --
Suggested Test Plan:
1) Find an account which has errata auto-updates disabled.
2) Create and publish a new errata.
3) Wait for the errata to get applied.
You can monitor the progress of errata processing by:
3a) Edit /usr/share/rhn/classes/log4j.properties on scripts.back-webqa.redhat.com
3b) Add the following line:
3c) Bounce taskomatic: /sbin/service taskomatic restart
3d) Monitor the /var/log/rhn/rhn_taskomatic_daemon.log file
4) Once errata processing is complete, verify that the server used in step #1
does not have the errata scheduled.
-- Additional comment from firstname.lastname@example.org on 2007-03-22 14:18 EST --
looks like its not fixed.
ksmith is already looking in to it.
-- Additional comment from email@example.com on 2007-03-22 16:40 EST --
Ok. Let's try this again with a different test path. We'll need to simulate the
effects of prod-ops' errata scripts so some manual database munging is involved.
1) Use the /svn/trunk/eng/backend/server/test/test_errata_import_webqa.py script
to create a new errata. Change the advisory name before running the script. I've
used the house number part of my address to try and insure uniqueness.
2) After the script has run, execute the following SQL query against the webqa
select id from rhnErrata where created > sysdate - 1 and created < sysdate and
advisory like '%ADVISORY_NAME%'
Note: Replace the term ADVISORY_NAME with the unique value you selected in step
1. Make sure to leave the percent signs in place otherwise the query will not
3) Jot down the id returned from the query.
4) Create a work entry for taskomatic so it can process the new errata. This is
normally done by prod-ops' scripts, but we'll have to simulate it here.
insert into rhnErrataQueue values (ERRATA_ID, sysdate, sysdate, sysdate);
Note: Replace the term ERRATA_ID with the id from step 3.
5) Wait for taskomatic to process this record. I'd suggest waiting 15-20 minutes.
6) Login to an account which has servers with auto errata updates disabled.
7) Verify that these servers _do not_ have a pending errata action for the newly
-- Additional comment from firstname.lastname@example.org on 2007-03-23 08:50 EST --
Thanks kevin for the detailed test plan.
-- Additional comment from email@example.com on 2007-03-26 09:41 EST --
I've checked this in stage using my own account...
System 1007229597 has a bunch of errata scheduled (correct)
System 1007229598 does *not* have a bunch of errata scheduled (correct)
-- Additional comment from firstname.lastname@example.org on 2007-03-26 09:49 EST --
Moving to RELEASE_PENDING
-- Additional comment from email@example.com on 2007-11-09 11:39 EST --
This problem appears to be back as I have most of my auto-apply systems yet to
apply RHEL5.1 .. Some SSIDs for ya:
(I have lots more if necessary)
What we have seen in the one customer case we know of:
- ~40 errata out of the ~136 RHEL 5.1 GA errata were successfully scheduled on
auto-errata update enabled systems.
- 4 asynchronous errata (openldap 6255, tetex 6254, pcre 6256, kdegraphics 6257)
released post 5.1 GA were correctly scheduled and updated.
- If rhn_check is run on an affected system, the 5.1 GA errata are not applied.
(and they are not queued up to be applied as 'pending actions' in the webui for
- it seems that if other actions are scheduled, and rhn_check is ran, those
actions are scheduled and take place without issue.
Created attachment 258171 [details]
attaching a customer-provided list of errata that did not get scheduled for the
Currently, for errata released today
samba RHSA-2007:1114 113 systems in "None" status for me
python RHSA-2007:1077 0 systems currently scheduled for it
sysreport RHBA-2007:1071 1 system in "Pending"
We are still waiting for RHEL5 and 4.6 to be fully scheduled for our account.
A data later and another data point.
samba RHSA-2007:1114 112 systems in "None" status for me
python RHSA-2007:1077 is OK now
sysreport RHBA-2007:1071 1 system in "Pending"
The 1 less for samba, is because I manually applied it on one machine as a test.
No meaningful change with the samba or sysreport errata.
Autofs and lvm2 errata's on 12 Dec seemed to get scheduled just fine.
Any updates from your side?
Still no meaningful change from last updates. Still waiting for samba,
sysreport, RHEL 5.1 and RHEL 4.6 to be scheduled.
Some script/action was run against our account last night scheduling all sorts
of errata against our "Auto-Errata" machines. Any comments on what was done?
It would appear that a script or action was run against Iowa State machines
again last night. Two days straight now, RHN has attempted to apply updates that
have already been applied for machines set to Auto Errata update.
Here's a line of the output of /var/log/up2date for the machine with system ID
[Tue Jan 8 19:57:31 2008] up2date RPM package conflict error. The message was:
Test install failed because of package conflicts:
package libpng-1.2.7-3.el4_5.1 is already installed
package libpng-devel-1.2.7-3.el4_5.1 is already installed
More alarming is an attempt to install a kernel (which I assume would be
prevented by the skiplist):
[Tue Jan 8 19:57:24 2008] up2date RPM package conflict error. The message was:
Test install failed because of package conflicts:
package kernel-smp-devel-2.6.9-67.0.1.EL (which is newer than kernel-smp-devel-2
.6.9-55.0.9.EL) is already installed
package kernel-hugemem-devel-2.6.9-67.0.1.EL (which is newer than kernel-hugemem
-devel-2.6.9-55.0.9.EL) is already installed
Created attachment 291159 [details]
Daily RHN Status emails
Daily status emails from 8 Jan and 9 Jan showing the many old errata getting
scheduled. All of these actions were scheduled around 7 PM on the 7th and 8th.
task-o-matic is running again this morning :) Lots of errata from 2007 getting
Created attachment 291287 [details]
RHN daily status report 10 Jan 2008
Today's RHN status email showing all sorts of errata getting scheduled.
But I only have 114 systems :)
Here is probably the best example of this problem:
Showing 9 of my systems in "None" status for redhat-release Errata included with
SSID: 1008254853 remains in its untouched state for you folks to diagnose if
Created attachment 292154 [details]
RHN Status Email for 18 Jan 2008
Got this status email this morning showing a couple hundred errata from 2007
getting queued. The only problem is that I can find no evidence of this on RHN
web UI under scheduled or completed actions.
We are still waiting for RHEL 4.6, RHEL 5.1, the samba errata and any sort of
explaination for what happened on 7 January. Is anybody at redhat looking at
this bugzilla? The only update we get from GSS is that there is no update to
I am very concerned that this bug isn't being addressed. This bug is not the
RHEL4 client + rhn satellite bug that you are feverishly working to fix.
If this bug is being worked on, what bugzilla entry contains the progress?
Daryl, this bug is clearly and correctly filed under RHN (hosted), not Satellite.
But where is the activity on this bug taking place? GSS told me there are a
number of private bz entries with lots of activity regarding this issue. I
highly doubt they are against RHN hosted. Those are sat customers.
Does red hat believe this issue and the rhel4 client on rhn satellite to be related?
Hi Daryl, I'm not sure where you are seeing a connection to a satellite bug in
this bug, but I don't see it; the private comments are very much hosted
specific. I do not know if they are thought to be related issues.
Okay, thanks as always for the help. I sure hope this is fixed before RHEL 5.2
Was this actually fixed in rhn 5.0.5 ? Or is it pending some other release?
The issue about 4.6/5.1 errata was fixed. I am going to open another BZ
regarding seamonkey not getting scheduled (BZ 441389). Dev is currently looking
at the process to push errata as it appears that some are not getting scheduled