Bug 1284037 - Traceback displayed on GoneException received from candlepin
Traceback displayed on GoneException received from candlepin
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: virt-who (Show other bugs)
6.8
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Radek Novacek
gaoshang
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-11-20 10:51 EST by Chris Snyder
Modified: 2016-11-30 19:32 EST (History)
4 users (show)

See Also:
Fixed In Version: virt-who-0.16-1.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-10 19:56:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Chris Snyder 2015-11-20 10:51:59 EST
Description of problem:
As a part of Bug 1271158
a traceback is shown when virt-who references a consumer that has been deleted.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Register a hypervisor to stage
2. Start virt-who
3. Delete the hypervisor using the stage candlepin site 
4. Look at rhsm.log 

Actual results:

2015-10-13 17:19:30,360 [ERROR]  @virtwho.py:190 - Error in communication with subscription manager:
Traceback (most recent call last):
  File "/usr/share/virt-who/virtwho.py", line 171, in send
    self._sendGuestList(report)
  File "/usr/share/virt-who/virtwho.py", line 196, in _sendGuestList
    manager.sendVirtGuests(report.guests)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 139, in sendVirtGuests
    self.connection.updateConsumer(self.uuid(), guest_uuids=serialized_guests)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 947, in updateConsumer
    ret = self.conn.request_put(method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 653, in request_put
    return self._request("PUT", method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 597, in validateResponse
    parsed['displayMessage'], parsed['deletedId'])

Expected results:
No stack trace. There should also be a log message stating that communication with rhsm failed because the consumer no longer exists.
Comment 3 Radek Novacek 2015-11-24 09:26:39 EST
The previous commit doesn't handle it very well, this one should be better:

https://github.com/virt-who/virt-who/commit/eb66da3d8f89307835dcb1cfbd5a715e4487d19e
Comment 4 Radek Novacek 2015-12-17 07:30:43 EST
Fixed by rebase to virt-who-0.16-1.el6.
Comment 6 Liushihui 2016-02-25 21:55:12 EST
Reopen it on virt-who-0.16-4.el6.noarch since virt-who cannot detect the hypervisor has been deleted also cannot send any error info in the log.

Rechecked version:
subscription-manager-1.16.8-2.el6.x86_64
python-rhsm-1.16.6-1.el6.x86_64
virt-who-0.16-3.el6.noarch

Rechecked process:
1 Register system to stage candlepin.
2 Configure virt-who monitor hyperv.
[root@sgi-xe500-01 run]# cat /etc/sysconfig/virt-who  | grep -v ^# | grep -v ^$
VIRTWHO_DEBUG=1
VIRTWHO_INTERVAL=2
VIRTWHO_HYPERV=1
VIRTWHO_HYPERV_OWNER=7715246
VIRTWHO_HYPERV_ENV=7715246
VIRTWHO_HYPERV_SERVER=10.73.5.212
VIRTWHO_HYPERV_USERNAME=administrator
VIRTWHO_HYPERV_PASSWORD=Welcome1
3 Restart virt-who 
[root@sgi-xe500-01 run]# service virt-who restart
4 Delete the system using the stage candlepin site
5 Check the virt-who log in rhsm.log
[root@sgi-xe500-01 run]# tail -f /var/log/rhsm/rhsm.log

Result:
virt-who cannot send log message stating that communication with rhsm failed because the consumer no longer exists. it repeat send normally info in the virt-who log as the following:
2016-02-25 20:56:25,935 [virtwho.env_cmdline DEBUG] HyperV-1(3085):MainThread @hyperv.py:retry_ntlm_negotitate:67 - Using NTLM authentication
2016-02-25 20:56:26,509 [virtwho.env_cmdline DEBUG] HyperV-1(3085):MainThread @hyperv.py:retry_ntlm_authenticate:80 - Sending NTLM authentication data
2016-02-25 20:56:26,810 [virtwho.env_cmdline DEBUG] HyperV-1(3085):MainThread @hyperv.py:retry_ntlm_authenticate:103 - NTLM authentication successful
2016-02-25 20:56:32,623 [virtwho.env_cmdline DEBUG] HyperV-1(3085):MainThread @virt.py:enqueue:351 - Report gathered, putting to queue for sending
2016-02-25 20:56:32,625 [virtwho.main INFO] MainProcess(3077):MainThread @virtwho.py:run:305 - Report for config "env/cmdline" haven't changed, not sending
Comment 7 Radek Novacek 2016-03-15 10:07:19 EDT
You have to do some change in host/guest association (start some VM), otherwise the report is not sent ("Report for config "env/cmdline" haven't changed, not sending") and this bug won't trigger.
Comment 8 Liushihui 2016-03-16 01:46:55 EDT
Checked it on virt-who-0.16-7.el6.noarch,when mapping info update, virt-who can show error info after delete system on satellite web UI, error info as the following:
2016-03-16 01:40:46,376 [virtwho.main ERROR] MainProcess(26864):MainThread @virtwho.py:send:203 - Unable to send data: Communication with subscription manager failed with code 500: undefined method `inject' for nil:NilClass

However, the error info is not clear, would you mind to stating that "communication with rhsm failed because the consumer no longer exists."
Comment 9 Radek Novacek 2016-03-17 03:40:22 EDT
This looks like you're facing some bug in candlepin. This error is returned by candlepin: "undefined method `inject' for nil:NilClass". What candlepin version do you use?
Comment 10 Liushihui 2016-03-30 02:34:23 EDT
Verified it on virt-who-0.16-7.el6.noarch against latest stage candlepin since it will state "Unable to send data: Communication with subscription manager failed: consumer no longer exists" if system has been delete from web UI when host/guest updating on hyperv/esx. Therefore, verify it.

Verified version:
virt-who-0.16-7.el6.noarch 
subscription-manager-1.16.8-7.el6.x86_64 
python-rhsm-1.16.6-1.el6.x86_64

Verified process:
1 Register system to stage candlepin.
2 Configure virt-who monitor hyperv.
[root@sgi-xe500-01 run]# cat /etc/sysconfig/virt-who  | grep -v ^# | grep -v ^$
VIRTWHO_DEBUG=1
VIRTWHO_INTERVAL=2
VIRTWHO_HYPERV=1
VIRTWHO_HYPERV_OWNER=7715246
VIRTWHO_HYPERV_ENV=7715246
VIRTWHO_HYPERV_SERVER=10.73.5.212
VIRTWHO_HYPERV_USERNAME=administrator
VIRTWHO_HYPERV_PASSWORD=Welcome1
3 Restart virt-who 
[root@sgi-xe500-01 run]# service virt-who restart
4 Delete the system using the stage candlepin site
5.In hyperv, start/stop/pause guests.
6 Check the virt-who log in rhsm.log
[root@sgi-xe500-01 run]# tail -f /var/log/rhsm/rhsm.log
2016-03-30 02:30:35,101 [virtwho.main INFO] MainProcess(12420):MainThread @subscriptionmanager.py:hypervisorCheckIn:185 - Sending update in hosts-to-guests mapping for config "env/cmdline": 1 hypervisors and 2 guests found
2016-03-30 02:30:35,101 [virtwho.main DEBUG] MainProcess(12420):MainThread @subscriptionmanager.py:hypervisorCheckIn:186 - Host-to-guest mapping: {
    "564D80F1-61F0-8510-87BF-57F9EC6122D7": [
        {
            "guestId": "C877953C-A19E-5149-B614-D97873C4CBD2", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "hypervisorVersion": "6.3.9600.16404", 
                "virtWhoType": "hyperv", 
                "hypervisorType": "hyperv"
            }
        }, 
        {
            "guestId": "8329390B-8359-F646-9176-27D738580E81", 
            "state": 5, 
            "attributes": {
                "active": 0, 
                "hypervisorVersion": "6.3.9600.16404", 
                "virtWhoType": "hyperv", 
                "hypervisorType": "hyperv"
            }
        }
    ]
}
2016-03-30 02:30:35,862 [virtwho.main ERROR] MainProcess(12420):MainThread @virtwho.py:send:203 - Unable to send data: Communication with subscription manager failed: consumer no longer exists
2016-03-30 02:30:35,863 [virtwho.main DEBUG] MainProcess(12420):MainThread @virtwho.py:send_report:168 - Report from "env/cmdline" failed to sent

Result:
No stack trace. There is a log message stating that "Communication with subscription manager failed: consumer no longer exists"
Comment 12 errata-xmlrpc 2016-05-10 19:56:55 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0859.html

Note You need to log in before you can comment on or make changes to this bug.