Bug 1286945

Summary: Properly handle http 429 error code
Product: Red Hat Enterprise Linux 7 Reporter: Radek Novacek <rnovacek>
Component: virt-whoAssignee: Radek Novacek <rnovacek>
Status: CLOSED ERRATA QA Contact: Eko <hsun>
Severity: unspecified Docs Contact: Yehuda Zimmerman <yzimmerm>
Priority: high    
Version: 7.2CC: ajanorka, carceneaux, hsun, ovasik, rnovacek, yzimmerm
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: virt-who-0.17-1.el7 Doc Type: Enhancement
Doc Text:
_virt-who_ handles HTTP error 429 properly When the Subscription Manager load is too big, it might return HTTP error code 429 to rate-limit communication with the client. Previously, _virt-who_ did not handle this error code properly, resulting in sub-optimal behavior. With this update, _virt-who_ now handles HTTP error code 429 properly and retries the communication with Subscription Manager later.
Story Points: ---
Clone Of:
: 1368365 (view as bug list) Environment:
Last Closed: 2016-11-04 05:07:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 1368365    

Description Radek Novacek 2015-12-01 07:43:50 UTC
Candlepin has recently introduced rate-limitation for host-guest associations. It returns HTTP error code 429 when the guests (including virt-who) are sending request too often. The clients should increase checking-in interval when they get this error code.

Unfortunately current version of virt-who in RHEL-7 can't handle this kind of reply and it fails with following error:

> 2015-11-18 15:11:29,132 [INFO]  @subscriptionmanager.py:124 - Sending update
> in hosts-to-guests mapping: {inferno.mitre.org:
> [42203c35-09bc-cf00-6d2a-c60b55231418, 42205d69-c1f5-cda7-d93d-131dbabf7a67,
> 42202abf-3785-791d-b000-12cabc19f92b, 42205f24-2... blah blah
> 2015-11-18 15:11:30,038 [ERROR]  @virtwho.py:123 - Error in communication
> with subscription manager:
> Traceback (most recent call last):
>   File "/usr/share/virt-who/virtwho.py", line 105, in send
>     self._sendGuestAssociation(report)
>   File "/usr/share/virt-who/virtwho.py", line 134, in _sendGuestAssociation
>     result = manager.hypervisorCheckIn(report.config, report.association,
>     report.config.type)
>   File
>   "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py",
>   line 134, in hypervisorCheckIn
>     return self.connection.hypervisorCheckIn(config.owner, config.env,
>     mapping)
>   File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 815, in
>   hypervisorCheckIn
>     return self.conn.request_post(url, host_guest_mapping)
>   File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 605, in
>   request_post
>     return self._request("POST", method, params)
>   File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 530, in
>   _request
>     self.validateResponse(result, request_type, handler)
>   File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 589, in
>   validateResponse
>     raise NetworkException(response['status'])
> NetworkException: Network error code: 429

Support for this error code has been added in virt-who upstream and needs to be backported to RHEL-7. This affects lots of customers.

Comment 1 Chris Arceneaux 2016-02-23 22:30:18 UTC
Is there somewhere I can find what the checking-in internal minimum is? I had mine set to 5 minutes previously. I've changed it to 1 hour just now to see if that resolves the issue but I'd really like the interval to be shorter than a hour.

Comment 2 Radek Novacek 2016-05-17 13:02:59 UTC
Fixed in virt-who-0.17-1.el7.

Comment 4 Eko 2016-07-19 09:02:18 UTC
verified with virt-who-0.17-6.el7.noarch

Comment 7 Radek Novacek 2016-09-29 14:35:47 UTC
There is a typo in the Doc Text: HTTOP -> HTTP, otherwise looks good. Thanks.

Comment 9 errata-xmlrpc 2016-11-04 05:07:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2387.html