Is taskomatic running on the box?

Yes, it is.

I just noticed something interesting.  The errata are not showing
applicable in the web interface, but I just got an email from the
satellite informing me that most of my 200 machines require the latest
kernerl errata.  This tells me that the satellite is aware of the
errata to machine mapping at some level, but it's not showing it to me
on the website so I can schedule it.

We definitely need to check to see if this is happening in latest
satellite code.

Ok, it looks like the erratacache is not being scheduled when a
satellite_sync is run.  We should be scheduling it for each channel
that satellite_sync touches.

How I reproduced it:

1. I took a (3.2) satellite that had not yet received the latest update.

2. I registered an AS3 system against it.  Two errata applied to the
system.

3. I ran 'satellite-sync' with no arguments.

4. When the sync was complete, I checked the errata page for the
system, and the same two errata apply.

5. I scheduled an errata cache update from
https://rhnblade4.perf.redhat.com/internal/satellite/task_engine/index.pxt

6. Now 56 errata apply to my system.

The sat is rhnblade4.perf.redhat.com - it is a 3.2 satellite, which is
the newest I have access to.

Re-assigning to taw - unless this is fixed in 3.3, please fix.

This is what the perl code does when a channel is modified:

SELECT task_data
  FROM rhnTaskQueue
 WHERE task_data = :cid


if rows are returned {
UPDATE rhnTaskQueue
   SET earliest = TO_DATE(:now, 'YYYY-MM-DD HH24:MI:SS') + :delay
 WHERE task_data = :cid

(:delay is '0' for immediate update)
}
else {
INSERT INTO rhnTaskQueue
       (org_id, task_name, task_data, priority, earliest)
VALUES (:org_id, 'update_errata_cache_by_channel', :cid, 0,
TO_DATE(:now, 'YYYY-MM-DD HH24:MI:SS') + :delay)
}

This sucks big time. Working on it.

Kevin... does the problem go away if you do a
satellite-sync --cache-refresh-level=6
?

Kambiz ran your satellite-sync command on Monday, and it did update
the errata counts for errata that had already been published.  I got
an errata update email this morning for the RHEL 3WS tcpdump errata
indicating that it was applicable to most of my machines, but the web
interface on the satellite only indicates it being applicable to 4
machines.

Short answer to your question: nope :(

I think this might be a RHN 260 issue. We fixed a major issue with
packages that reside in multiple channels. It was not really an issue
until RHEL3 where it reared it's ugly head.

I recommend scheduling an upgrade.

Oh... also NOTE. I was *not* able to reproduce this in RHN 320. I
think rnorwood is on crack! :)

Can you confirm something for me?

Does the problem go away if you:
satellite-sync --list-channels # to get the labels
Sync each channel individually:
satellite-sync --cache-refresh-level=6 -c CHANNEL_LABEL

I don't know if this will fix it, but we'll see.
I am going to get an RHN 260 box set up to try to duplicate all of this.

Ok... I think this is CURRENTRELEASE but I want QA to take a look at this.

TEST PLAN:
----------
o install RHN Satellite
o sync RHEL 3 WS
  *then*
  sync RHEL 3 WS, ES and AS
o register a client machine to each distro. The client machines need
  to have the some of the same errata needed for each, so making them
  all QU1 should be fine.
o verify that all relavent errata are available for each of these   
  clients... and reported as needed by the clients.

CHANGE OF TEST PLAN:
--------------------
o install RHN Satellite
o service taskomatic stop
o sync RHEL 3 WS
  *then*
  sync RHEL 3 WS, ES and AS
o register a client machine to each distro. The client machines need
  to have the some of the same errata needed for each, so making them
  all QU1 should be fine.
o sqlplus USERNAME/PASSWORD@SID
  select * from rhnTaskQueue;
  ...there should be a list of channels there for WS, ES, and AS
o service taskomatic start
o verify that all relavent errata are available for each of these   
  clients... and reported as needed by the clients.

I received the error below while trying to satellite-sync the RHEL 3
WS channel to my satellite (https://farm03.rhndev.redhat.com):

Exception Handler Information
Traceback (innermost last):
  File "/usr/bin/satellite-sync", line 129, in main
    return satsync.main()
  File "/var/www/rhns/satellite_tools/satsync.py", line 1709, in main
    syncer.processErrata()
  File "/var/www/rhns/satellite_tools/satsync.py", line 770, in
processErrata
    processErrata()
  File "/var/www/rhns/satellite_tools/syncContainerHandlers.py", line
371, in processErrata
    importer.run()
  File "/var/www/rhns/server/importlib/importLib.py", line 515, in run
    self.fix()
  File "/var/www/rhns/server/importlib/errataImport.py", line 110, in fix
    self._fix_erratum_packages(erratum)
  File "/var/www/rhns/server/importlib/errataImport.py", line 232, in
_fix_erratum_packages
    for nevrao, package in erratum['packages'].items():
AttributeError: items

I can't duplicate that error as much as I have tried. Everything seems
to work for me. It was possibly environmental? Regardless, I am
kicking  this back to ON_QA for testing.

If you encounter this problem again: stop testing, and immediately
fetch me to troubleshoot the system directly.

Copying part of the discussion I had with Jamo:
The fix:
satellite-sync --cache-refresh-level=6

What I think is going on: I think you are switching environments (webdev
to webqa) without blowing away your sync cache. What this does is leave
all kinds of cruft sitting in the cache that doesn't match what is in
hosted. An example: kernel erratum X has a bunch of packages with
incorrect IDs in it, etc.

So... that traceback is NOTABUG IMHO. We need to document this issue
though as a test-environment concern.
-Todd

The fix did _not_ work.  That is, satellite-sync
--cache-refresh-level=6 did not do the trick.

Granted, this was on satellite code based on 2.6, and now we are
running version 3.2.0.

Kevin, can you verify if this problem has gone away, or if it still
stands since we upgraded the satellite to 3.2.0?

Kambiz

This require a lot of time to test "syncing channels"...  Deferring to
rhn360sat since it is not critical.

this problem seems to have disappeared in RHN Sat 3.6. I am closing
this bug CURRENTRELEASE and we simply have to keep on a lookout for
this problem resurfacing. - Todd Warner (sitting at Fanny's desk :)