Bug 1002007
| Summary: | proxy "Exception type exceptions.KeyError" and "XMLRPC ProtocolError" | |||
|---|---|---|---|---|
| Product: | [Community] Spacewalk | Reporter: | Sebastien Foucou <admin.systeme> | |
| Component: | Proxy Server | Assignee: | Michael Mráka <mmraka> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Red Hat Satellite QA List <satqe-list> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 2.0 | CC: | admin.systeme, cperry, lpramuk, mmraka | |
| Target Milestone: | --- | |||
| Target Release: | --- | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | spacewalk-proxy-2.0.2-1 | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1002441 (view as bug list) | Environment: | ||
| Last Closed: | 2013-12-19 11:58:43 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 991452 | |||
The broker is not broking,
/etc/httpd/error_log:
Exception Handler Information
Traceback (most recent call last):
File "/usr/share/rhn/proxy/apacheServer.py", line 62, in __call__
ret = f(req)
File "/usr/share/rhn/proxy/apacheHandler.py", line 348, in handler
handlerObj = rhnBroker.BrokerHandler(req)
File "/usr/share/rhn/proxy/broker/rhnBroker.py", line 70, in __init__
my_ip_addr = req.headers_in['SERVER_ADDR']
KeyError: 'SERVER_ADDR'
Oops. I meant /var/log/httpd/error_log: Lukas, can you tell me specifically what you did to get that error? client -> proxy (el5) -> satellite [@client]# rhnreg_ks --username=$USER --password=$PASS --serverUrl=https://$PROXY/XMLRPC --sslCACert=/usr/share/rhn/${PROXY}-TRUSTED-SSL-CERT --force An error has occurred: Error communicating with server. The message was: Internal Server Error See /var/log/up2date for more information /var/log/up2date:
...
File "/usr/share/rhn/up2date_client/rhnserver.py", line 160, in __get_capabilities
self.registration.welcome_message()
File "/usr/share/rhn/up2date_client/rhnserver.py", line 51, in __call__
return rpcServer.doCall(method, *args, **kwargs)
File "/usr/share/rhn/up2date_client/rpcServer.py", line 278, in doCall
raise up2dateErrors.CommunicationError(e.errmsg)
<class 'up2date_client.up2dateErrors.CommunicationError'>: Error communicating with server. The message was:
Internal Server Error
I changed proxy spec to use mod_wsgi on RHEL5 but there are still some issues; investigating difference between RHEL6 and RHEL5...
commit 34c3e9c2c0177345a8f0c0ce2a2a9757e136ec67
1002007 - use mod_wsgi even on RHEL5
Fixed python 2.4 httplib issues in Spacewalk master:
commit dafdc3d2245f8f8ec02ddbe92be316a3ab2505a8
Automatic commit of package [spacewalk-proxy] release [2.1.9-1].
commit cd5f85734304214dc40470b5e4ef6b96c89ffc47
1002007 - don't send empty data
addressing
Traceback (most recent call last):
File "/usr/share/rhn/proxy/apacheServer.py", line 62, in __call__
ret = f(req)
File "/usr/share/rhn/proxy/apacheHandler.py", line 355, in handler
ret = handlerObj.handler()
File "/usr/share/rhn/proxy/redirect/rhnRedirect.py", line 92, in handler
status = self._serverCommo() # part 2
File "/usr/share/rhn/proxy/rhnShared.py", line 201, in _serverCommo
status, headers, bodyFd = self._proxy2server()
File "/usr/share/rhn/proxy/rhnShared.py", line 370, in _proxy2server
http_connection.send(data)
File "/usr/lib64/python2.4/httplib.py", line 664, in send
self.sock.sendall(str)
File "/usr/lib/python2.4/site-packages/rhn/SSL.py", line 217, in write
sent = self._connection.send(data)
SysCallError: (-1, 'Unexpected EOF')
Oops, pylint fix
commit eaaa9d48395d4b1b706ee0b48d366f66e9926fda
fixed pylint error
Hi, I started this morning an update proxy First, i installed and activated the mod_wsgi module for apache Second, i installed the proxy update The installed version is repository "nightly": spacewalk-ssl-cert-check-2.3-1.el5 spacewalk-proxy-broker-2.1.10-1.el5 spacewalk-proxy-docs-2.0.1-1.el5 spacewalk-setup-jabberd-2.0.1-1.el5 spacewalk-base-minimal-config-2.1.16-1.el5 spacewalk-proxy-common-2.1.10-1.el5 spacewalk-proxy-management-2.1.10-1.el5 spacewalk-monitoring-selinux-2.0.1-1.el5 spacewalk-proxy-html-2.0.1-1.el5 spacewalk-proxy-selinux-2.0.1-1.el5 spacewalk-certs-tools-2.1.1-1.el5 spacewalk-base-minimal-2.1.16-1.el5 spacewalk-backend-2.1.14-1.el5 spacewalk-proxy-redirect-2.1.10-1.el5 spacewalk-proxy-monitoring-2.0.1-1.el5 spacewalk-backend-libs-2.1.14-1.el5 spacewalk-proxy-installer-2.1.1-1.el5 spacewalk-proxy-package-manager-2.1.10-1.el5 I unsubscribed the proxy server on spacewalk : # rhn-proxy-activate --deactivate I started the proxy configuration: # /usr/sbin/configure-proxy.sh I did a test from a client and everything works perfectly now. Thank you very much for your help! NB: the spacewalk server remained in version 2.0, do you recommend an update to 2.1? (In reply to Sebastien Foucou from comment #9) > Hi, > > I started this morning an update proxy ... > I did a test from a client and everything works perfectly now. > > Thank you very much for your help! > > NB: the spacewalk server remained in version 2.0, do you recommend an update > to 2.1? Hello Sebastien, Thank you for reporting back. Please do not upgrade your server, current nightly version is not supposed to be stable and there might not be an easy way to upgrade to final 2.1 once it will be released. Conversely we feel this is a serious bug and we work on backporting it to current 2.0 proxy version. Regards, Michael The fixed version spacewalk-proxy-2.0.2-1 has been released as a part of Spacewalk 2.0 release. If the solution does not work for you, please open a new bug report. |
Description of problem: We upgraded Spacewalk to version 2.0. the master instance looks fine and everything looks fine. But we have problems after upgrade of the proxy instance in our datacenter. Now all servers behind the proxy are not able to query the master instance with error: “XMLRPC ProtocolError: <ProtocolError for ***proxy*** /XMLRPC: 500 Internal Server Error>” Version-Release number of selected component (if applicable): CentOS release 5.9 (Final) x86_64 spacewalk-ssl-cert-check-2.3-1.el5 spacewalk-base-minimal-2.0.3-1.el5 spacewalk-proxy-installer-2.0.1-1.el5 spacewalk-proxy-package-manager-2.0.1-1.el5 spacewalk-proxy-redirect-2.0.1-1.el5 spacewalk-backend-libs-2.0.3-1.el5 spacewalk-proxy-docs-2.0.1-1.el5 spacewalk-backend-2.0.3-1.el5 spacewalk-setup-jabberd-2.0.1-1.el5 spacewalk-monitoring-selinux-2.0.1-1.el5 spacewalk-proxy-html-2.0.1-1.el5 spacewalk-proxy-common-2.0.1-1.el5 spacewalk-proxy-selinux-2.0.1-1.el5 spacewalk-certs-tools-2.0.1-1.el5 spacewalk-proxy-monitoring-2.0.1-1.el5 spacewalk-proxy-broker-2.0.1-1.el5 spacewalk-proxy-management-2.0.1-1.el5 rhncfg-5.10.55-1.el5 rhncfg-management-5.10.55-1.el5 yum-rhn-plugin-2.0.1-1.el5 rhncfg-actions-5.10.55-1.el5 rhn-check-2.0.1-1.el5 rhn-client-tools-2.0.1-1.el5 rhncfg-client-5.10.55-1.el5 rhnsd-5.0.13-1.el5 rhnlib-2.5.65-1.el5 rhnpush-5.5.65-1.el5 rhn-setup-2.0.1-1.el5 rhn-org-httpd-ssl-key-pair-spwkf1p1-1.0-11 How reproducible: doing "yum repolist" on client system Actual results: Mail report from proxy server : Exception reported from spwkf1p1 Time: Tue Aug 27 16:06:45 2013 Exception type exceptions.KeyError Exception while handling function handler Request object information: URI: /XMLRPC Remote Host: xxx.xxx.xxx.xxx Server Name: spwkf1p1.secret.local:443 Headers passed in: Accept-Encoding: identity Content-Length: 2709 Host: spwkf1p1.secret.local content-type: text/xml user-agent: rhn.rpclib.py/$Revision$ x-client-version: 1 x-info: RPC Processor (C) Red Hat, Inc (version $Revision$) x-rhn-client-capability: packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1 x-rhn-transport-capability: follow-redirects=3 x-transport-info: Extended Capabilities Transport (C) Red Hat, Inc (version $Revision$) x-up2date-version: 1.7.14-1.el6 Extra information about this error: Unhandled exception type Exception Handler Information Traceback (most recent call last): File "/usr/share/rhn/proxy/apacheServer.py", line 62, in __call__ ret = f(req) File "/usr/share/rhn/proxy/apacheHandler.py", line 355, in handler ret = handlerObj.handler() File "/usr/share/rhn/proxy/broker/rhnBroker.py", line 215, in handler status = self._serverCommo() # part 2 File "/usr/share/rhn/proxy/rhnShared.py", line 201, in _serverCommo status, headers, bodyFd = self._proxy2server() File "/usr/share/rhn/proxy/rhnShared.py", line 366, in _proxy2server http_connection.send(self.req.headers_in['wsgi.input']) KeyError: 'wsgi.input' Local variables by frame Frame _proxy2server in /usr/share/rhn/proxy/rhnShared.py at line 366 http_connection = <type 'instance'> <rhn.connections.HTTPSConnection instance at 0x2aefa4f55200> hdrs = <type 'instance'> {'content-length': '2709', 'x-client-version': '1', 'accept-encoding': 'identity', 'x-rhn-proxy-version': '5.5.0', 'x-transport-info': 'Extended Capabilities Transport (C) Red Hat, Inc (version $Revision$)', 'x-up2date-version': '1.7.14-1.el6', 'user-agent': 'rhn.rpclib.py/$Revision$', 'x-rhn-ip-path': ' xxx.xxx.xxx.xxx', 'host': 'spwkf1p1.secret.local', 'x-rhn-transport-capability': 'follow-redirects=3', 'x-rhn-proxy-auth': '1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local', 'x-info': 'RPC Processor (C) Red Hat, Inc (version $Revision$)', 'content-type': 'text/xml', 'x-rhn-client-capability': 'packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.ver ify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1'} k = <type 'str'> X-RHN-Proxy-Auth v = <type 'str'> 1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local vals = <type 'list'> ['1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local'] self = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38> size = <type 'int'> 2709 Frame _serverCommo in /usr/share/rhn/proxy/rhnShared.py at line 214 self = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38> Frame handler in /usr/share/rhn/proxy/broker/rhnBroker.py at line 215 authToken = <type 'str'> 1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local client_ip = <type 'str'> xxx.xxx.xxx.xxx self = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38> _oto = <type 'instance'> {'content-length': '2709', 'x-client-version': '1', 'accept-encoding': 'identity', 'x-rhn-proxy-version': '5.5.0', 'x-transport-info': 'Extended Capabilities Transport (C) Red Hat, Inc (version $Revision$)', 'x-up2date-version': '1.7.14-1.el6', 'user-agent': 'rhn.rpclib.py/$Revision$', 'x-rhn-ip-path': ' xxx.xxx.xxx.xxx', 'host': 'spwkf1p1.secret.local', 'x-rhn-transport-capability': 'follow-redirects=3', 'x-rhn-proxy-auth': '1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local', 'x-info': 'RPC Processor (C) Red Hat, Inc (version $Revision$)', 'content-type': 'text/xml', 'x-rhn-client-capability': 'packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.ver ify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1'} getResult = <type 'NoneType'> None tokens = <type 'list'> ['1000010002::1377594702.78:21600.0:1ti06mIfYFzXS4uChdC2Mg==:spwkf1p1.secret.local'] ip_path = <type 'str'> xxx.xxx.xxx.xxx _i = <type 'int'> 0 Frame handler in /usr/share/rhn/proxy/apacheHandler.py at line 357 handlerObj = <type 'instance'> <proxy.broker.rhnBroker.BrokerHandler instance at 0x2aefa4e4ef38> self = <type 'instance'> <proxy.apacheHandler.apacheHandler instance at 0x2aefa4d9bb48> req = <type 'mp_request'> <mp_request object at 0x2aefa4ade050> ret = <type 'int'> 0 rhnBroker = <type 'module'> <module 'proxy.broker.rhnBroker' from '/usr/share/rhn/proxy/broker/rhnBroker.pyc'> Frame __call__ in /usr/share/rhn/proxy/apacheServer.py at line 68 self = <type 'instance'> <proxy.apacheServer.HandlerWrap instance at 0x2aefa4d0d908> req = <type 'mp_request'> <mp_request object at 0x2aefa4ade050> f = <type 'instancemethod'> <bound method apacheHandler.handler of <proxy.apacheHandler.apacheHandler instance at 0x2aefa4d9bb48>> Frame HandlerDispatch in /usr/lib64/python2.4/site-packages/mod_python/apache.py at line 299 req = <type 'mp_request'> <mp_request object at 0x2aefa4ade050> config = <type 'mp_table'> {'PythonPath': 'sys.path+['/usr/share/rhn']'} self = <type 'instance'> <mod_python.apache.CallBack instance at 0x2aefa4ad6bd8> object = <type 'instance'> <proxy.apacheServer.HandlerWrap instance at 0x2aefa4d0d908> l = <type 'list'> ['proxy.apacheServer', 'Handler'] module = <type 'module'> <module 'proxy.apacheServer' from '/usr/share/rhn/proxy/apacheServer.pyc'> hlist = <type 'mp_hlist'> {'handler:'proxy.apacheServer::Handler','directory':'/XMLRPC/','silent':0} object_str = <type 'str'> Handler debug = <type 'int'> 0 module_name = <type 'str'> proxy.apacheServer pathstring = <type 'str'> sys.path+['/usr/share/rhn'] result = <type 'int'> 500 Environment for PID=16456 on exception: LANG = C PATH = /sbin:/usr/sbin:/bin:/usr/bin PWD = / SHLVL = 2 TERM = xterm _ = /usr/sbin/httpd Expected results: Additional info: Database : Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production