Bug 1002441 - proxy "Exception type exceptions.KeyError" and "XMLRPC ProtocolError"
proxy "Exception type exceptions.KeyError" and "XMLRPC ProtocolError"
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite Proxy 5
Classification: Red Hat
Component: Server (Show other bugs)
560
x86_64 Linux
unspecified Severity urgent
: ---
: ---
Assigned To: Michael Mráka
Lukas Pramuk
:
Depends On:
Blocks: sat560-blockers sat560-proxy
  Show dependency treegraph
 
Reported: 2013-08-29 04:29 EDT by Michael Mráka
Modified: 2013-10-01 15:40 EDT (History)
1 user (show)

See Also:
Fixed In Version: spacewalk-proxy-2.0.1-8
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1002007
Environment:
Last Closed: 2013-10-01 15:40:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Michael Mráka 2013-08-29 04:29:16 EDT
+++ This bug was initially created as a clone of Bug #1002007 +++

Description of problem:

We upgraded Spacewalk to version 2.0. the master instance looks fine and everything looks fine.

But we have problems after upgrade of the proxy instance in our datacenter.

Now all servers behind the proxy are not able to query the master instance with error: “XMLRPC ProtocolError: <ProtocolError for ***proxy*** /XMLRPC: 500 Internal Server Error>”

Version-Release number of selected component (if applicable):

CentOS release 5.9 (Final)
x86_64

spacewalk-ssl-cert-check-2.3-1.el5
spacewalk-base-minimal-2.0.3-1.el5
spacewalk-proxy-installer-2.0.1-1.el5
spacewalk-proxy-package-manager-2.0.1-1.el5
spacewalk-proxy-redirect-2.0.1-1.el5
spacewalk-backend-libs-2.0.3-1.el5
spacewalk-proxy-docs-2.0.1-1.el5
spacewalk-backend-2.0.3-1.el5
spacewalk-setup-jabberd-2.0.1-1.el5
spacewalk-monitoring-selinux-2.0.1-1.el5
spacewalk-proxy-html-2.0.1-1.el5
spacewalk-proxy-common-2.0.1-1.el5
spacewalk-proxy-selinux-2.0.1-1.el5
spacewalk-certs-tools-2.0.1-1.el5
spacewalk-proxy-monitoring-2.0.1-1.el5
spacewalk-proxy-broker-2.0.1-1.el5
spacewalk-proxy-management-2.0.1-1.el5
rhncfg-5.10.55-1.el5
rhncfg-management-5.10.55-1.el5
yum-rhn-plugin-2.0.1-1.el5
rhncfg-actions-5.10.55-1.el5
rhn-check-2.0.1-1.el5
rhn-client-tools-2.0.1-1.el5
rhncfg-client-5.10.55-1.el5
rhnsd-5.0.13-1.el5
rhnlib-2.5.65-1.el5
rhnpush-5.5.65-1.el5
rhn-setup-2.0.1-1.el5
rhn-org-httpd-ssl-key-pair-spwkf1p1-1.0-11

How reproducible:

doing "yum repolist" on client system

Actual results:

Mail report from proxy server : 

Exception reported from spwkf1p1
Time: Tue Aug 27 16:06:45 2013
Exception type exceptions.KeyError
Exception while handling function handler Request object information:
URI: /XMLRPC
Remote Host:  xxx.xxx.xxx.xxx
Server Name: spwkf1p1.secret.local:443
Headers passed in:
	Accept-Encoding: identity
	Content-Length: 2709
	Host: spwkf1p1.secret.local
	content-type: text/xml
	user-agent: rhn.rpclib.py/$Revision$
	x-client-version: 1
	x-info: RPC Processor (C) Red Hat, Inc (version $Revision$)
	x-rhn-client-capability: packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1
	x-rhn-transport-capability: follow-redirects=3
	x-transport-info: Extended Capabilities Transport (C) Red Hat, Inc (version $Revision$)
	x-up2date-version: 1.7.14-1.el6
Extra information about this error:
Unhandled exception type

Exception Handler Information
Traceback (most recent call last):
  File "/usr/share/rhn/proxy/apacheServer.py", line 62, in __call__
    ret = f(req)
  File "/usr/share/rhn/proxy/apacheHandler.py", line 355, in handler
    ret = handlerObj.handler()
  File "/usr/share/rhn/proxy/broker/rhnBroker.py", line 215, in handler
    status = self._serverCommo()       # part 2
  File "/usr/share/rhn/proxy/rhnShared.py", line 201, in _serverCommo
    status, headers, bodyFd = self._proxy2server()
  File "/usr/share/rhn/proxy/rhnShared.py", line 366, in _proxy2server
    http_connection.send(self.req.headers_in['wsgi.input'])
KeyError: 'wsgi.input'

Local variables by frame
Frame _proxy2server in /usr/share/rhn/proxy/rhnShared.py at line 366
	     http_connection = <type 'instance'> <rhn.connections.HTTPSConnection instance at 0x2aefa4f55200>
	                hdrs = <type 'instance'> {'content-length': '2709', 'x-client-version': '1', 'accept-encoding': 'identity', 'x-rhn-proxy-version': '5.5.0', 'x-transport-info': 'Extended Capabilities Transport (C) Red Hat, Inc (version $Revision$)', 'x-up2date-version': '1.7.14-1.el6', 'user-agent': 'rhn.rpclib.py/$Revision$', 'x-rhn-ip-path': ' xxx.xxx.xxx.xxx', 'host': 'spwkf1p1.secret.local', 'x-rhn-transport-capability': 'follow-redirects=3', 'x-rhn-proxy-auth': '1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local', 'x-info': 'RPC Processor (C) Red Hat, Inc (version $Revision$)', 'content-type': 'text/xml', 'x-rhn-client-capability': 'packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.ver
 ify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1'}
	                   k = <type 'str'> X-RHN-Proxy-Auth
	                   v = <type 'str'> 1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local
	                vals = <type 'list'> ['1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local']
	                self = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38>
	                size = <type 'int'> 2709

Frame _serverCommo in /usr/share/rhn/proxy/rhnShared.py at line 214
	                self = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38>

Frame handler in /usr/share/rhn/proxy/broker/rhnBroker.py at line 215
	           authToken = <type 'str'> 1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local
	           client_ip = <type 'str'>  xxx.xxx.xxx.xxx
	                self = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38>
	                _oto = <type 'instance'> {'content-length': '2709', 'x-client-version': '1', 'accept-encoding': 'identity', 'x-rhn-proxy-version': '5.5.0', 'x-transport-info': 'Extended Capabilities Transport (C) Red Hat, Inc (version $Revision$)', 'x-up2date-version': '1.7.14-1.el6', 'user-agent': 'rhn.rpclib.py/$Revision$', 'x-rhn-ip-path': ' xxx.xxx.xxx.xxx', 'host': 'spwkf1p1.secret.local', 'x-rhn-transport-capability': 'follow-redirects=3', 'x-rhn-proxy-auth': '1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local', 'x-info': 'RPC Processor (C) Red Hat, Inc (version $Revision$)', 'content-type': 'text/xml', 'x-rhn-client-capability': 'packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.ver
 ify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1'}
	           getResult = <type 'NoneType'> None
	              tokens = <type 'list'> ['1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local']
	             ip_path = <type 'str'>  xxx.xxx.xxx.xxx
	                  _i = <type 'int'> 0

Frame handler in /usr/share/rhn/proxy/apacheHandler.py at line 357
	          handlerObj = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38>
	                self = <type 'instance'> <proxy.apacheHandler.apacheHandler instance at 0x2aefa4d9bb48>
	                 req = <type 'mp_request'> <mp_request object at 0x2aefa4ade050>
	                 ret = <type 'int'> 0
	           rhnBroker = <type 'module'> <module 'proxy.broker.rhnBroker' from '/usr/share/rhn/proxy/broker/rhnBroker.pyc'>

Frame __call__ in /usr/share/rhn/proxy/apacheServer.py at line 68
	                self = <type 'instance'> <proxy.apacheServer.HandlerWrap instance at 0x2aefa4d0d908>
	                 req = <type 'mp_request'> <mp_request object at 0x2aefa4ade050>
	                   f = <type 'instancemethod'> <bound method apacheHandler.handler of <proxy.apacheHandler.apacheHandler instance at 0x2aefa4d9bb48>>

Frame HandlerDispatch in /usr/lib64/python2.4/site-packages/mod_python/apache.py at line 299
	                 req = <type 'mp_request'> <mp_request object at 0x2aefa4ade050>
	              config = <type 'mp_table'> {'PythonPath': 'sys.path+['/usr/share/rhn']'}
	                self = <type 'instance'> <mod_python.apache.CallBack instance at 0x2aefa4ad6bd8>
	              object = <type 'instance'> <proxy.apacheServer.HandlerWrap instance at 0x2aefa4d0d908>
	                   l = <type 'list'> ['proxy.apacheServer', 'Handler']
	              module = <type 'module'> <module 'proxy.apacheServer' from '/usr/share/rhn/proxy/apacheServer.pyc'>
	               hlist = <type 'mp_hlist'> {'handler:'proxy.apacheServer::Handler','directory':'/XMLRPC/','silent':0}
	          object_str = <type 'str'> Handler
	               debug = <type 'int'> 0
	         module_name = <type 'str'> proxy.apacheServer
	          pathstring = <type 'str'> sys.path+['/usr/share/rhn']
	              result = <type 'int'> 500


Environment for PID=16456 on exception:
LANG = C
PATH = /sbin:/usr/sbin:/bin:/usr/bin
PWD = /
SHLVL = 2
TERM = xterm
_ = /usr/sbin/httpd



Expected results:


Additional info:

Database :
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
Comment 1 Michael Mráka 2013-08-30 09:56:40 EDT
Backported to SATELLITE-5.6 as
commit 78fad924601f57aefa28d7761ded5a171828cb03
    fixed pylint error
commit fa66f0676024f69c8fa2391e66119009064e62d9
    1002007 - don't send empty data
commit 9264a21d5e6ebfc83d1791732f169963d5d83d7e
    1002007 - python 2.4 HTTPConnection can't read directly from object
commit 0979c3bc86336871ce0fdb38a0ad8133bc935685
    1002007 - use mod_wsgi even on RHEL5
Comment 2 Michael Mráka 2013-08-30 09:57:34 EDT
Fix is included in compose RHN-Proxy-5.6.0-RHEL5-re20130830.0, moving ON_QA.
Comment 4 Clifford Perry 2013-10-01 15:40:09 EDT
Red Hat Satellite Proxy 5.6 has been released. This bug tracked under Proxy. 

This bug either was VERIFIED or RELEASE_PENDING (re-verified prior shortly
before release). 

Moving to CLOSED CURRENT_RELEASE. 

Text from Upgrade Erratum follows:



Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2013-1392.html

Note You need to log in before you can comment on or make changes to this bug.