Bug 61463 - scheduled actions complete but RHN still says needs updating
scheduled actions complete but RHN still says needs updating
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: up2date (Show other bugs)
4.0
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Adrian Likins
Jay Turner
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2002-03-19 20:22 EST by James Manning
Modified: 2015-01-07 18:55 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2002-03-22 09:17:46 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
up2date log showing first check-in since 59 errata scheduled (1.52 KB, text/plain)
2002-03-20 13:04 EST, James Manning
no flags Details

  None (edit)
Description James Manning 2002-03-19 20:22:06 EST
Pretty simple - machine has 2 errata out on it, they're selected and updating them
is scheduled, the time comes, they happen, the packages update fine, but
RHN doesn't reflect it (aside from showing the action completed) - it still says
the machine needs those 2 errata.  Manually running up2date -p works, as
would (I imagine) using the RHN interface for refreshing the pkg list.

I've confirmed this with 3 machines so far - looks like the scheduled actions stuff
just needs to do the pkg-refresh logic as well and currently doesn't.

Haven't checked the manual up2date -u case.

If this is an rhnsd bug, just let me know and I'll refile under RHL/up2date
Comment 1 Adrian Likins 2002-03-19 22:19:52 EST
which errata?

I'm not seeing this on my work or home systems, and didnt see
it during testing, so I'm a bit confused as to whats going on.

I'm getting proper package refreshes with errata and package updates.

what systemid is this for? can you attach a copy of the /etc/sysconfig/rhn/up2date ?
Comment 2 Adrian Likins 2002-03-19 22:23:47 EST
whats the history item for these actions say?

Do you get the same behaviour for scheduled package updates?

Comment 3 James Manning 2002-03-20 10:51:20 EST
ok, machine lvs01, RH 7.1, sid 1000676071

RHN web site says 59 errata 154 pkgs apply to it. 

I clicked on the system name, clicked the "big red button" of "Update This 
System" 

System last checkin: 2002-03-20 09:24:42 -0500 (EST) 
Current RHN time: 2002-03-20 10:18:53 -0500 (EST) 
Expected checkin time: 2002-03-20 11:24:42 -0500 (EST) 

and now my Pending Action list has 59 entries in it.  I'll update this bugzilla 
entry after the update happens (RHN says I have no more pending and the 
completed ones were all successful) and I'll see what RHN says about system 
state after that.

Thanks, Adrian!
Comment 4 James Manning 2002-03-20 12:40:19 EST
ok, the system is still largely out of date, but the first checkin happened and 
the up2date stuff updated and rhnsd started (looks like I'll have to wait for 
the next checkin for everything else).

What's interesting is that RHN still says 59 errata and 154 pkgs need to be 
applied to this system *and* scheduled actions still says 59 pending/in 
progress.

This seems wrong given that RHBA-2002:044-12 (the recent up2date push) has 
definitely been pushed out as an errata.  This could be an unrelated bug where 
the rhnsd restart prematurely kills off rhnsd before RHN seems that the clients 
got installed?  Either that or updating the up2date stuff is just considered 
some kind of core refresh and doesn't count as an action completing, even it if 
happens to match an errata.

[admin@lvs01 log]$ rpm -q python-xmlrpc rhn_register rhn_register-gnome up2date 
up2date-gnome python-popt
python-xmlrpc-1.5.1-7.x.3
rhn_register-2.7.9-7.x.2
rhn_register-gnome-2.7.9-7.x.2
up2date-2.7.46-7.x.1
up2date-gnome-2.7.46-7.x.1
python-popt-0.6-4

I'll attach the /var/log/up2date just in case it helps
Comment 5 James Manning 2002-03-20 13:04:39 EST
Created attachment 49225 [details]
up2date log showing first check-in since 59 errata scheduled
Comment 6 Adrian Likins 2002-03-20 13:47:30 EST
hmm, the rhnsd restarting sounds like a possibility...

Kind of a odd case, and something I could of missed
in testing... (I was generally `up2date up2date` and then
installing every package, scheduleing every erratat, and
scheduleing every package, so it's possible I missed the
update up2date action...)

digging in...
Comment 7 James Manning 2002-03-21 16:12:34 EST
ok, I've tracked down an instance that should provide a case.

https://rhn.redhat.com/network/system/system_history_event.pxt?sid=1000658815&hid=2328573

That shows netsaint getting upgraded for RHSA-2002:026-39

The current action status is: Completed
The client picked up this action on 2002-03-21 05:20:56 -0500 (EST)
The client reported completion on execution on 2002-03-21 05:21:12 -0500 (EST)
Client execution returned code 0 (Packages were installed successfully)

Indeed, the update did happen:

[root@netsaint /root]# rpm -q zlib
zlib-1.1.3-25.6
(this matches the number in the errata)

PROBLEM: RHN still says netsaint needs the errata:

https://rhn.redhat.com/network/system/system_errata_list.pxt?sid=1000658815
Comment 8 James Manning 2002-03-21 16:13:46 EST
note - this isn't just the zlib vulnerability case - there's 106 actions that
completed against netsaint, but RHN still says 106 errata are needed for it
Comment 9 Adrian Likins 2002-03-21 16:26:39 EST
I see that is's happening but at the moment, I dont know why...

investigating...
Comment 10 Adrian Likins 2002-03-21 17:23:12 EST
Interesting.

I see "installing packages foo" in the up2date log, but no info
about upgrading the package profile. 

Can you apply the equilivent of:

--- up2date.py  11 Mar 2002 21:47:10 -0000      1.263
+++ up2date.py  21 Mar 2002 22:17:49 -0000
@@ -2189,6 +2189,8 @@
     removedPackages = []
     addedPackages = []
 
+    print "olderinstalledPackages: %s\n\n\n" % oldInstalledPackages
+    print "newInstalledPackagesL %s\n\n\n" % newInstalledPackages
     # find packages that were removed
     for pkg in oldInstalledPackages:
         if pkg not in newInstalledPackages:


And try updating an errata. You'll need to redirect that to
a file, since the file lists will be pretty big.

I'm suspecting that for some reason the query to get the list
of currently installed is giving bogus results. 

also, try this seperately... copy your existing rpmdb somewhere, then
try "rpm --rebuilddb". If it's getting bogus values for the
installedpackagelist, that could be why. 

I would think that would show up elsewhere, but I'm shooting in the dark here...
Comment 11 James Manning 2002-03-22 00:16:48 EST
I can't take exactly that approach, at least AFAICT.  As noted elsewhere (I may 
have forgotten to mention this in this particular bugzilla entry - they're all 
blurring together for me now :) if I use up2date everything appears to work 
fine - the package update happens, the errata count drops, etc.

At the moment, I'm actually trying to figure out the code path that gets to 
up2date.updatePackageProfile at all - I see how -p gets the "packages" option 
set and then on an if of that, it gets called, but when I grep on the files 
coming out of rpm -ql up2date, I don't see anything else calling it.

Well, that notwithstanding, I can make the requested mod, but since the broken 
path is when it's a scheduled action (not called via rhnsd), i'll have to do it 
as log.log_me calls but that should be ok.

Since all the packages are actually upgraded for netsaint, I'm gonna take the 
new imlib errata and try to reproduce this again:

- machine needs errata (confirm with rpm -q and post here)
- errata update gets scheduled (post info here)
- machine gets updated (RHN sees action completed, rpm -q sees update)
- RHN still says errata needed (with url)

when I find a machine/errata combo that looks happy for getting the above going 
I'll toss in the 2 new log entries (that'll dump a ton of text in there) and 
then after it's all over I'll toss in the attachments here.
Comment 12 Adrian Likins 2002-03-22 00:29:29 EST
In the case of errata updates it more or less
      errata.update
          packages.update
              batchRun
                 wrapper.batchRun
                     up2date.installPackages
                         up2date.remoteAddPackages and/or
                         up2date.remoteDelPackages 

(ie, it's a bit deep down there. My kingdom for time to rewrite that... ieee)


If you add the debugging mentioned, and just "rhn_check -v" you should
see the output from the action being delivered and ran
Comment 13 James Manning 2002-03-22 00:42:39 EST
[oh, ok, that makes sense - i was looking for a updatePackageProfile call, but 
what you said makes more sense]

ok, bp6 (system id 1000675709) needs the new php package - see 
https://rhn.redhat.com/network/system/system_outdated_package_list.pxt?
sid=1000675709

I've added the log.log_me lines as requested:

--- up2date.py.orig     Fri Mar 22 00:31:38 2002
+++ up2date.py  Fri Mar 22 00:27:35 2002
@@ -2189,6 +2189,9 @@
     removedPackages = []
     addedPackages = []

+    log.log_me("olderinstalledPackages: %s\n\n\n" % oldInstalledPackages)
+    log.log_me("newInstalledPackagesL %s\n\n\n" % newInstalledPackages)
+
     # find packages that were removed
     for pkg in oldInstalledPackages:
         if pkg not in newInstalledPackages:

and i've just scheduled the php errata to update on bp6

System last checkin: 2002-03-21 23:15:09 -0500 (EST) 
Current RHN time: 2002-03-22 00:34:11 -0500 (EST) 
Expected checkin time: 2002-03-22 01:15:09 -0500 (EST) 

i'll post with the results after 1:15 or in the morning
Comment 14 James Manning 2002-03-22 00:46:41 EST
forgot to post this earlier - it's the "prove the machine needs the errata" step

jmm@bp6:/usr/share/rhn/up2date_client> rpm -q php
php-4.0.6-12
Comment 15 Adrian Likins 2002-03-22 00:49:51 EST
you can just run:

      env -i rhn_check -v -v -v


Thats what gets called from rhnsd
(minus cleansing the enviroment, which is another bug, soon
to be fixed (aieee...))
Comment 16 James Manning 2002-03-22 01:10:36 EST
I went ahead and scheduled the imlib errata too so it may provide another case

jmm@bp6:/usr/share/rhn/up2date_client> rpm -q imlib
imlib-1.9.10-2

i'll go ahead and just wait for the 1:15 run since it's about to happen anyway
Comment 17 James Manning 2002-03-22 01:32:15 EST
well, the time came and the scheduled actions failed because of another (I 
believe you've tracked down the cause on this one?) bug:

The current action status is: Failed
The client picked up this action on 2002-03-22 01:15:34 -0500 (EST)
The client reported completion on execution on 2002-03-22 01:15:43 -0500 (EST)
Client execution returned code 25 (Failed: The package signing key for Red Hat, 
Inc. is not on the gpg keyring)

when i reschedule the php errata and then do your rhn_check:

[root@bp6 log]#       env -i /usr/sbin/rhn_check -v -v -v
check_action({'version': 2, 'id': 2370430, 'action': "<?xml version='1.0'?
>\012<methodCall>\012<methodName>errata.update</methodName>\012<params>\012<para
m>\012<value><array><data>\012<value><int>1029</int></value>\012</data></array><
/value>\012</param>\012</params>\012\012</methodCall>\012"})
handle_action({'version': 2, 'id': 2370430, 'action': "<?xml version='1.0'?
>\012<methodCall>\012<methodName>errata.update</methodName>\012<params>\012<para
m>\012<value><array><data>\012<value><int>1029</int></value>\012</data></array><
/value>\012</param>\012</params>\012\012</methodCall>\012"})
handle_action actionid = 2370430, version = 2
do_call(errata.update, ([1029],))
Your GPG keyring does not contain the Red Hat, Inc. public key.
Without it, you will be unable to verify that packages Update Agent downloads
are securely signed by Red Hat.

Your Update Agent options specify that you want to use GPG.

To install the key, run the following as root:

    /usr/bin/gpg --no-default-key --keyring /etc/sysconfig/rhn/up2date-
keyring.gpg --import /usr/share/rhn/RPM-GPG-KEY
Sending back response: (25, 'Failed: The package signing key for Red Hat, Inc. 
is not on the gpg keyring', {})


PROBLEM #1: the key is already in there

[root@bp6 log]# gpg --list-keys --no-default-keyring --
keyring /etc/sysconfig/rhn/up2date-keyring.gpg
/etc/sysconfig/rhn/up2date-keyring.gpg
--------------------------------------
pub  1024D/DB42A60E 1999-09-23 Red Hat, Inc <security@redhat.com>
sub  2048g/961630A2 1999-09-23

PROBLEM #2:  the listed command gets one of the gpg flags wrong - the correct 
param is --no-default-keyring, not --no-default-key
Comment 18 Adrian Likins 2002-03-22 01:38:30 EST
the keyring error I've tracked down and fixed (and ugh, what a pain...
we pulled this errata in the time being)

problem #2: This was a typo in the help message, and is fixed
now (it should of been building the text up from the function that
builds the gpg args, but wasnt). I'ts fixed now as well.

I'm going to bounce you a copy of the new packages, can you
try them out?

Comment 19 James Manning 2002-03-22 02:42:06 EST
YES YES YES YES YES!

I'll test whatever you're willing to throw at me :)
Comment 20 James Manning 2002-03-22 08:21:43 EST
sweet!  I scheduled the php errata again on bp6 and then did the the env 
rhn_check command you specified with up2date-2.7.60-7.x.2 and it worked like a 
champ!  downloaded, installed, and RHN caught the update!

Since bp6 only had the 1 errata left to apply (the imlib errata), I clicked the 
big red "update this system" button and it scheduled that one action and I'm 
gonna let rhnsd do the rhn_check spawn this time just to make sure there wasn't 
some silly regression there (highly doubtful).  When it works, I'll 
close/rawhide this bug if that's ok :)

Awesome work, Adrian!!
Comment 21 Adrian Likins 2002-03-22 09:17:42 EST
Excellent. Thanks for testing those out.

If you are in the mood for testing, feel free to fiddle around
with the gui/cli stuff. 

Some of the changes I had to make could potentially have impact
on those areas. I think I got everything worked out, but it
can be hard to tell.
Comment 22 James Manning 2002-05-15 15:26:53 EDT
long-since fixed

Note You need to log in before you can comment on or make changes to this bug.