Bug 1167238
| Summary: | Error in communication with subscription manager | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Liushihui <shihliu> | ||||
| Component: | virt-who | Assignee: | Radek Novacek <rnovacek> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Li Bin Liu <liliu> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 7.1 | CC: | alikins, gxing, ovasik, sgao, shihliu, wpoteat | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | All | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | virt-who-0.14-2.el7 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-11-19 11:55:51 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1121117 | ||||||
| Attachments: |
|
||||||
Created attachment 960629 [details]
rhsm.log
And some error log as the following:
2014-11-24 11:37:16,177 [ERROR] @virtwho.py:134 - Error in communication with subscription manager, trying to recover:
Traceback (most recent call last):
File "/usr/share/virt-who/virtwho.py", line 126, in _send
self._sendGuests(virt, virtualGuests)
File "/usr/share/virt-who/virtwho.py", line 152, in _sendGuests
manager.sendVirtGuests(virtualGuests)
File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 97, in sendVirtGuests
self._connect()
File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 75, in _connect
if not self.connection.ping()['result']:
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 725, in ping
return self.conn.request_get("/status/")
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 570, in request_get
return self._request("GET", method)
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 498, in _request
self.validateResponse(result, request_type, handler)
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 540, in validateResponse
raise RestlibException(response['status'], error_msg)
RestlibException: The proxy server received an invalid response from an upstream server
Looks like the problem is higher in the stack, reassigning to python-rhsm for investigation. please provide an rhsm-debug file. sudo rhsm-debug system Use the latest rhel7.1 and stage candlepin, do the same test as bug's description, it hasn't reproduced. Checked version: virt-who-0.11-5.el7.noarch subscription-manager-1.13.22-1.el7.x86_64 python-rhsm-1.13.10-1.el7.x86_64 Latest stage candelpin (In reply to Liushihui from comment #3) > And some error log as the following: > 2014-11-24 11:37:16,177 [ERROR] @virtwho.py:134 - Error in communication > with subscription manager, trying to recover: > Traceback (most recent call last): > File "/usr/share/virt-who/virtwho.py", line 126, in _send > self._sendGuests(virt, virtualGuests) > File "/usr/share/virt-who/virtwho.py", line 152, in _sendGuests > manager.sendVirtGuests(virtualGuests) > File > "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", > line 97, in sendVirtGuests > self._connect() > File > "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", > line 75, in _connect > if not self.connection.ping()['result']: > File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 725, in > ping > return self.conn.request_get("/status/") > File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 570, in > request_get > return self._request("GET", method) > File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 498, in > _request > self.validateResponse(result, request_type, handler) > File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 540, in > validateResponse > raise RestlibException(response['status'], error_msg) > RestlibException: The proxy server received an invalid response from an > upstream server That looks like what I'd expect python-rhsm to do if it got a 502 error from the server. For reasons unknown to me, stage is very prone to throwing 502 errors, and those will end up as rhsm.connection.RestlibExceptions. So virt-who needs to understand how to handle those. Or more generically, rhsm.connection.ConnectionException that is the base class of RestlibException So bouncing back to virt-who virt-who didn't failed to operate when it saw this kind of error, it just dropped the report it tried to post. I've added check for these kind of error and virt-who will try to resend the report a couple of times. It will still print the backtraces for this error to the log file because it can't know if it sends incorrect data or there is only some temporary error. This change is present in virt-who-0.14-2.el7. Verified it on virt-who-0.14-7.el7 since it won't generate RestlibException. Therefore, verify it. Verified version: virt-who-0.14-7.el7.noarch subscription-manager-1.15.9-7.el7.x86_64 python-rhsm-1.15.4-4.el7.x86_64 Latest stage candlepin Verified process: 1. Register system to stage candlepin 2. Configure virt-who run at default mode(libvirt) 3. Check the virt-who log: # tail -f /var/log/rhsm/rhsm.log Result: It hasn't show any RestlibException in the log file. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2370.html |
Description of problem: When system have registered to stage candlepin, virt-who will show many error message when communicate with subscription-manager Version-Release number of selected component (if applicable): subscription-manager-1.13.8-1.el7.x86_64 python-rhsm-1.13.7-1.el7.x86_64 virt-who-0.11-3.el7.noarch How reproducible: 50% Steps to Reproduce: 1. Register to stage candlepin with stage_virtwho_test_oct/redhat 2. Configure virt-who run at default mode(libvirt) 3. Check the virt-who log: # tail -f /var/log/rhsm/rhsm.log Actual results: It will show error log as the following, please see detail in the attachment 2014-11-24 09:25:45,667 [ERROR] @virtwho.py:134 - Error in communication with subscription manager, trying to recover: Traceback (most recent call last): File "/usr/share/virt-who/virtwho.py", line 126, in _send self._sendGuests(virt, virtualGuests) File "/usr/share/virt-who/virtwho.py", line 152, in _sendGuests manager.sendVirtGuests(virtualGuests) File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 112, in sendVirtGuests self.connection.updateConsumer(self.uuid(), guest_uuids=domains) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 815, in updateConsumer ret = self.conn.request_put(method, params) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 579, in request_put return self._request("PUT", method, params) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 498, in _request self.validateResponse(result, request_type, handler) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 540, in validateResponse raise RestlibException(response['status'], error_msg) RestlibException: Runtime Error Guice provision errors: Expected results: It shouldn't show any error exception in the rhsm log. Additional info: It hasn't this problem when system registered to SAM