Bug 1337398

Summary: virt-who failed to send h/g mapping when work at libvirt mode as "undefined method `[]=' for nil:NilClass"
Product: Red Hat Satellite Reporter: Liushihui <shihliu>
Component: Content ManagementAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: urgent    
Version: 6.2.0CC: gxing, hsun, jentrena, sgao, sreber, tomckay, xdmoon
Target Milestone: UnspecifiedKeywords: Regression
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-24 16:36:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1115190    

Description Liushihui 2016-05-19 06:34:31 UTC
Description of problem:
When virt-who run at default mode(libvirt mode), virt-who failed to send h/g mapping info to server

Version-Release number of selected component (if applicable):
Satellite-6.2.0-RHEL-7-20160518
virt-who-0.17-1.el7.noarch
subscription-manager-1.17.6-1.el7.x86_64
python-rhsm-1.17.2-1.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Register system to satellite6.2
2.Make virt-who run at default mode:
[root@hp-dl2x170g6-01 ~]# cat /etc/sysconfig/virt-who  | grep -v ^# | grep -v ^$
VIRTWHO_DEBUG=1
3.Restart virt-who service and check virt-who's log.
[root@hp-dl2x170g6-01 ~]# service virt-who restart && tail -f /var/log/rhsm/rhsm.log 
.......................
2016-05-19 02:31:44,939 [virtwho.main INFO] MainProcess(30922):MainThread @subscriptionmanager.py:sendVirtGuests:149 - Sending update in guests lists for config "env/cmdline": 0 guests found
2016-05-19 02:31:44,939 [virtwho.main DEBUG] MainProcess(30922):MainThread @subscriptionmanager.py:sendVirtGuests:150 - Domain info: []
2016-05-19 02:31:47,814 [virtwho.main ERROR] MainProcess(30922):MainThread @executor.py:send:156 - Error in communication with subscription manager:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/virtwho/executor.py", line 137, in send
    self._sendGuestList(report)
  File "/usr/lib/python2.7/site-packages/virtwho/executor.py", line 162, in _sendGuestList
    manager.sendVirtGuests(report, self.options)
  File "/usr/lib/python2.7/site-packages/virtwho/manager/subscriptionmanager/subscriptionmanager.py", line 154, in sendVirtGuests
    self.connection.updateConsumer(self.uuid(), guest_uuids=serialized_guests, hypervisor_id=report.hypervisor_id)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 995, in updateConsumer
    ret = self.conn.request_put(method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 690, in request_put
    return self._request("PUT", method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 598, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 648, in validateResponse
    raise RestlibException(response['status'], error_msg, response.get('headers'))
RestlibException: undefined method `[]=' for nil:NilClass
2016-05-19 02:31:47,815 [virtwho.main DEBUG] MainProcess(30922):MainThread @executor.py:send_report:108 - Report from "env/cmdline" failed to sent


Actual results:
Virt-who failed to send host/guest mapping info to server when work at libvirt mode.

Expected results:
Virt-who should send h/g mapping info successfully to server when work at libvirt mode

Additional info:

Comment 1 Radek Novacek 2016-05-19 06:41:46 UTC
RestlibException: undefined method `[]=' for nil:NilClass

It seems to be an issue in candlepin, reassigning...

Comment 2 Julio Entrena Perez 2016-05-19 15:52:36 UTC
Getting the following trace using virt-who-0.17-1.el7 with type=vdsm :

# virt-who -o -d
2016-05-19 16:38:14,535 [virtwho.init DEBUG] MainProcess(3168):MainThread @executor.py:__init__:65 - Using config named 'potato'
2016-05-19 16:38:14,535 [virtwho.init INFO] MainProcess(3168):MainThread @main.py:main:157 - Using configuration "potato" ("vdsm" mode)
2016-05-19 16:38:14,535 [virtwho.init INFO] MainProcess(3168):MainThread @main.py:main:159 - Using reporter_id='host1.example.com-d725f23b80dc4a79bf5b81e9f41780e0'
2016-05-19 16:38:14,598 [virtwho.potato DEBUG] Vdsm-1(3174):MainThread @virt.py:run:364 - Virt backend 'potato' started
2016-05-19 16:38:14,847 [virtwho.potato DEBUG] Vdsm-1(3174):MainThread @virt.py:enqueue:357 - Report for config "potato" gathered, putting to queue for sending
2016-05-19 16:38:14,848 [virtwho.potato DEBUG] Vdsm-1(3174):MainThread @virt.py:run:383 - Virt backend 'potato' stopped after sending one report
2016-05-19 16:38:14,873 [virtwho.main DEBUG] MainProcess(3168):MainThread @subscriptionmanager.py:_connect:123 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2016-05-19 16:38:15,452 [virtwho.main INFO] MainProcess(3168):MainThread @subscriptionmanager.py:sendVirtGuests:149 - Sending update in guests lists for config "potato": 0 guests found
2016-05-19 16:38:15,452 [virtwho.main DEBUG] MainProcess(3168):MainThread @subscriptionmanager.py:sendVirtGuests:150 - Domain info: []
2016-05-19 16:38:16,536 [virtwho.main ERROR] MainProcess(3168):MainThread @executor.py:send:156 - Error in communication with subscription manager:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/virtwho/executor.py", line 137, in send
    self._sendGuestList(report)
  File "/usr/lib/python2.7/site-packages/virtwho/executor.py", line 162, in _sendGuestList
    manager.sendVirtGuests(report, self.options)
  File "/usr/lib/python2.7/site-packages/virtwho/manager/subscriptionmanager/subscriptionmanager.py", line 154, in sendVirtGuests
    self.connection.updateConsumer(self.uuid(), guest_uuids=serialized_guests, hypervisor_id=report.hypervisor_id)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 947, in updateConsumer
    ret = self.conn.request_put(method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 653, in request_put
    return self._request("PUT", method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 611, in validateResponse
    raise RestlibException(response['status'], error_msg, response.get('headers'))
RestlibException: undefined method `[]=' for nil:NilClass
2016-05-19 16:38:16,538 [virtwho.main DEBUG] MainProcess(3168):MainThread @executor.py:send_report:108 - Report from "potato" failed to sent
2016-05-19 16:38:16,540 [virtwho.main DEBUG] MainProcess(3168):MainThread @__main__.py:main:19 - virt-who terminated
2016-05-19 16:38:16,541 [virtwho.main DEBUG] MainProcess(3168):MainThread @executor.py:terminate:303 - virt-who is shutting down

Is this the same problem or should I file a separate BZ for that?

Comment 3 Simon Reber 2016-05-20 09:44:54 UTC
I checked that further and it seems that the problem is that the host is not properly subscribed. In candlepin we have the following error:

2016-05-20 11:38:32,073 [thread=http-bio-8443-exec-4] [req=3ec2d30f-4dca-4187-806a-f7c5bc95b862, org=CGI_NET] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=795
2016-05-20 11:38:32,601 [thread=http-bio-8443-exec-9] [req=56df8ba9-3377-42dc-9b51-4c1bdaa1624c, org=CGI_NET] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=204, content-type="null", time=556
2016-05-20 11:38:32,671 [thread=http-bio-8443-exec-3] [req=5964aa53-78e5-4ff1-bd34-bad0ae20106f, org=] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=POST, uri=/candlepin/consumers/bbd3fb38-5298-4bbf-9c6e-693de480a0af/entitlements
2016-05-20 11:38:32,833 [thread=http-bio-8443-exec-3] [req=5964aa53-78e5-4ff1-bd34-bad0ae20106f, org=CGI_NET] INFO  org.candlepin.resource.ConsumerResource - Checked if consumer has unaccepted subscription terms in 0ms
2016-05-20 11:38:33,937 [thread=http-bio-8443-exec-3] [req=5964aa53-78e5-4ff1-bd34-bad0ae20106f, org=CGI_NET] INFO  org.candlepin.policy.js.autobind.AutobindRules - No pools available to rules for products: [8aa1105c54cd19100154cd74dcf504f9, 8aa1105c54cd19100154cd74dcf504fa]
2016-05-20 11:38:33,937 [thread=http-bio-8443-exec-3] [req=5964aa53-78e5-4ff1-bd34-bad0ae20106f, org=CGI_NET] INFO  org.candlepin.controller.CandlepinPoolManager - No entitlements available for products: [8aa1105c54cd19100154cd74dcf504f9, 8aa1105c54cd19100154cd74dcf504fa]
2016-05-20 11:38:33,970 [thread=http-bio-8443-exec-3] [req=5964aa53-78e5-4ff1-bd34-bad0ae20106f, org=CGI_NET] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=1299


The system is RHEL hypervisor for RHEV and I have attached all the subscription. But Satellite 6.2 does still not recognize the system as properly subscribed

Comment 4 Simon Reber 2016-05-20 12:57:39 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1335148 may fix that issue.

Comment 5 Eko 2016-05-23 09:13:05 UTC
this issue can be reproduced with virt-who-0.16-8.el6.noarch in rhel6.8

Comment 6 Barnaby Court 2016-05-23 15:30:12 UTC
Moving out the Candlepin component as the fix will have to be in the API proxy layer.