+++ This bug was initially created as a clone of Bug #867544 +++ Description of problem: rhn proxy 5.5 is connected to rhn hosted. after the x-rhn-auth-expire-offset expires a 2nd token is created but the headers continue to send the older token and client receives 404 for yum updates. [root@proxy proxy-auth]# pwd /var/cache/rhn/proxy-auth [root@proxy proxy-auth]# ll total 12 -rw-r--r--. 1 apache apache 313 Oct 17 16:32 1023707104 -rw-r--r--. 1 apache apache 75 Oct 11 15:11 p10237051934b84b15bff6ee5796152495a230e45e3d7e947d9 -rw-r--r--. 1 apache apache 86 Oct 17 11:32 p1023705193786cdd299f613e013b848b1f533206191f79cb03 [root@proxy proxy-auth]# cat p10237051934b84b15bff6ee5796152495a230e45e3d7e947d9 | cut -f5 -d: VSfX5ykdIFpMzfnR42c+Dw== [root@proxy proxy-auth]# cat p1023705193786cdd299f613e013b848b1f533206191f79cb03 | cut -f5 -d: 28dcyr8FNBzmnDGILz56sA== proxy/rhnShared._proxy2server('Outgoing header', 'X-RHN-Proxy-Auth', '1023705193::1349968279.16:21600.0:VSfX5ykdIFpMzfnR42c+Dw==:127.0.0.1') workaround is to delete older token in a cron job. customers are using cron jobs to wipe out /var/cache/rhn/proxy-auth We have one case where this is affecting satellite but I have yet been able to duplicate in house. My current proxy-> hosted reproducer is on my home external network for both the proxy and client. Version-Release number of selected component (if applicable): How reproducible: everytime on external network going through hosted but sometimes i have to wait 3600 minutes for the expire to take effect --- Additional comment from shughes on 2012-10-17 14:03:34 EDT --- so looks like the two different tokens is that one is for the ipaddress and one is for the hostname of the proxy: [root@proxy logs]# grep "token for this" rhn_proxy_broker.log 2012/10/17 17:40:08 -04:00 3862 192.168.1.22: broker/rhnBroker.handler('Auth token for this machine only! 1023705193::1350473526.3:21600.0:28dcyr8FNBzmnDGILz56sA==:proxy.hugheshoney.com',) 2012/10/17 17:40:09 -04:00 3861 192.168.1.22: broker/rhnBroker.handler('Auth token for this machine only! 1023705193::1350495611.41:21600.0:/Hm7xFxJ0268yx1xj2nLqQ==:proxy.hugheshoney.com',) 2012/10/17 17:40:10 -04:00 3855 192.168.1.22: broker/rhnBroker.handler('Auth token for this machine only! 1023705193::1350495611.41:21600.0:/Hm7xFxJ0268yx1xj2nLqQ==:proxy.hugheshoney.com',) 2012/10/17 17:40:13 -04:00 3860 127.0.0.1: broker/rhnBroker.handler('Auth token for this machine only! 1023705193::1349968279.16:21600.0:VSfX5ykdIFpMzfnR42c+Dw==:127.0.0.1',) 2012/10/17 17:40:13 -04:00 3856 192.168.1.22: broker/rhnBroker.handler('Auth token for this machine only! 1023705193::1350495611.41:21600.0:/Hm7xFxJ0268yx1xj2nLqQ==:proxy.hugheshoney.com',) 2012/10/17 17:40:14 -04:00 3861 127.0.0.1: broker/rhnBroker.handler('Auth token for this machine only! 1023705193::1349968279.16:21600.0:VSfX5ykdIFpMzfnR42c+Dw==:127.0.0.1',) what is interesting in the above yum update transaction is that we have two separate hashes for the hostname: 1023705193::1350473526.3:21600.0:28dcyr8FNBzmnDGILz56sA==:proxy.hugheshoney.com 1023705193::1350495611.41:21600.0:/Hm7xFxJ0268yx1xj2nLqQ==:proxy.hugheshoney.com
I cannot for the life of me figure out what has changed recently that causes this to break. There are several things that are very similar, but in each case closer inspection shows that they are not responsible for this error. However, we do have a fix. I believe that the commit soon to follow completely prevents this from happening. Note for QA: The only known reliable way to reproduce this error is to set up a client that is talking to a Proxy 5.5 that is talking to RHN Hosted, do a yum install of something on the client, wait 3 days or so, and then try to install something else on the client. It has been reported when the proxy is talking to Satellite too, but that is not 100% reproducible.
Committed to Spacewalk master: 5f00038675c58c5a401d0c4a5cbfa332c28a4405
just a note I am testing the fix with kickstarts. initial kickstart failed but looking into the error more closely. sherr, this is the configuration: client.hugheshoney.com -> proxy.hugheshoney.com -> hosted
On the Client system, yum will often return a 404 error, similar to the one below. Adding this comment, so that anyone doing BZ searches for client side error will hopefully find this BZ. Error Downloading Packages: yum-rhn-plugin-1.8.8-1.el6.noarch: failed to retrieve getPackage/yum-rhn-plugin-1.8.8-1.el6.noarch.rpm from centos-6-x86_64-spacewalk-client error was [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404" From spacewalk-list thread: "fix in the following commit: http://git.fedorahosted.org/cgit/spacewalk.git/commit/?id=5f00038675c58c5a401d0c4a5cbfa332c28a4405 After applying the patch locally my proxy is working as expected again. Yum update is finally working again. " Cliff
Marking bug as ON_QA since tonight's build of Spacewalk nightly is a release candidate for Spacewalk 1.9.
Spacewalk 1.9 has been released. https://fedorahosted.org/spacewalk/wiki/ReleaseNotes19