Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1348267

Summary: 500 Internal Server Error when virt-who attempts to post Hypervisors to Satellite 6.1.9
Product: Red Hat Satellite Reporter: Wilson Harris <wharris>
Component: CandlepinAssignee: Chris Snyder <csnyder>
Status: CLOSED ERRATA QA Contact: Perry Gagne <pgagne>
Severity: high Docs Contact:
Priority: high    
Version: UnspecifiedCC: abalakht, bbuckingham, bcourt, bkearney, csnyder, ehelms, hsun, jcallaha, kbidarka, mlele, mmccune, mstead, ovasik, sghai, wharris, xdmoon
Target Milestone: UnspecifiedKeywords: Reopened, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: candlepin-0.9.54.7-1,virt-who-0.17-10 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1349491 1405620 (view as bug list) Environment:
Last Closed: 2018-02-21 16:54:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1361491    
Bug Blocks:    
Attachments:
Description Flags
Foreman-debug from customer's satellite (running virt-who) none

Description Wilson Harris 2016-06-20 15:42:07 UTC
Created attachment 1169941 [details]
Foreman-debug from customer's satellite (running virt-who)

Description of problem:

When virt-who attempts to post hypervisor information onto satellite 6.1.9 there is a 500 internal server error. We can see in the rhsm.log where the connection to VMware works and the hypervisor information is sent. The error seems to occur when trying to post to the satellite.

Version-Release number of selected component (if applicable):

virt-who-0.14-9.el7.noarch

Below is the information I have gathered from the customer end. Customer information scrubbed. I have confirmed that all of the passwords inside the configuration files are correct. Below you will see (rhsm.log, virt-who, virt-who.d/qua-esx) It is also worthy to note that the satellite server itself is running virt-who.

rhsm.log

2016-06-09 09:05:57,965 [ERROR]  @subscriptionmanager.py:171 - Communication with server failed:
314883 Traceback (most recent call last):
314884   File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 167, in hypervisorCheckIn
314885     result = self.connection.hypervisorCheckIn(config.owner, config.env, serialized_mapping)
314886   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 903, in hypervisorCheckIn
314887     res = self.conn.request_post(url, host_guest_mapping)
314888   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 647, in request_post
314889     return self._request("POST", method, params)
314890   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
314891     self.validateResponse(result, request_type, handler)
314892   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 617, in validateResponse
314893     handler=handler)
314894 RemoteServerException: Server error attempting a POST to /rhsm/hypervisors?owner=default&env=Library returned status 500
314895 2016-06-09 09:05:57,966 [ERROR]  @virtwho.py:177 - Unable to send data: Communication with subscription manager failed with code 500: Server        error attempting a POST to /rhsm/hypervisors?owner=SAQ&env=Library returned status 500
314896 2016-06-09 09:05:57,968 [DEBUG]  @subscriptionmanager.py:108 - Authenticating with RHSM username admin
314897 2016-06-09 09:05:58,293 [DEBUG]  @subscriptionmanager.py:146 - Checking if server has capability 'hypervisor_async'
314898 2016-06-09 09:05:58,518 [DEBUG]  @subscriptionmanager.py:158 - Server does not have 'hypervisors_async' capability
314899 2016-06-09 09:05:58,563 [INFO]  @subscriptionmanager.py:165 - Sending update in hosts-to-guests mapping: {
314900     "server.esx.host": [
314901         {
314902             "guestId": "42307267-6649-6fb3-7597-e8d9ce3fdbfb",
314903             "state": 1,                                                                                              
...
319421 2016-06-09 09:06:02,444 [ERROR]  @subscriptionmanager.py:171 - Communication with server failed:
319422 Traceback (most recent call last):
319423   File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 167, in hypervisorCheckIn
319424     result = self.connection.hypervisorCheckIn(config.owner, config.env, serialized_mapping)
319425   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 903, in hypervisorCheckIn
319426     res = self.conn.request_post(url, host_guest_mapping)
319427   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 647, in request_post
319428     return self._request("POST", method, params)
319429   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
319430     self.validateResponse(result, request_type, handler)
319431   File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 617, in validateResponse
319432     handler=handler)
319433 RemoteServerException: Server error attempting a POST to /rhsm/hypervisors?owner=Default&env=Library returned status 500
319434 2016-06-09 09:06:02,445 [ERROR]  @virtwho.py:177 - Unable to send data: Communication with subscription manager failed with code 500: Server        error attempting a POST to /rhsm/hypervisors?owner=SAQ&env=Library returned status 500
319435 2016-06-09 09:06:02,445 [ERROR]  @virtwho.py:305 - Sending data failed 5 times, report skipped    

We can check the foreman/production.log to see where the 500 internal server error happens:

Line 4819 production.log

 </pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/7.0.54 logs.</u></p><HR size="1" noshade=     "noshade"><h3>Apache Tomcat/7.0.54</h3></body></html> (POST /candlepin/hypervisors?owner=SAQ&env=Library)
4820 Body: <html><head><title>Apache Tomcat/7.0.54 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-     color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:T     ahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-     color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:     white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 -      </h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The server encou     ntered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>java.lang.NullPointerException
4821         org.candlepin.common.exceptions.mappers.RollbackExceptionMapper.toResponse(RollbackExceptionMapper.java:55)
4822         org.candlepin.common.exceptions.mappers.RollbackExceptionMapper.toResponse(RollbackExceptionMapper.java:36)
                            
See the following lines for full traceback located inside latest foreman-debug provided by customer.

Virt-who Configuration File

# Enviromental variables for virt-who service can be specified here.
#
# These enviromental variables are only used when starting virt-who as service,
# otherwise you must specify them manually.
#
# Note that if some value contains special character, it must be escapted
# or the value must be quoted - for example ampersand in the password.

# Enable debugging output.
VIRTWHO_DEBUG=1

# Send the list of guest IDs and exit immediately.
#VIRTWHO_ONE_SHOT=0

# Acquire and send list of virtual guest each N seconds, 0 means default
# configuration.
VIRTWHO_INTERVAL=3600

# Virt-who subscription manager backend. Enable only one option from the following:
# Report to Subscription Asset Manager (SAM) or the Red Hat Customer Portal
# VIRTWHO_SAM=1
# Report to Sattellite version 6
VIRTWHO_SATELLITE6=1
# Report to Satellite verision 5
# VIRTWHO_SATELLITE5=0
# Report to Satellite [Legacy]
# VIRTWHO_SATELLITE=0


# Following configuration is now deprecated. Please use configuration file
# in /etc/virt-who.d/. See virt-who-config(5) for details how to configure it.

# virt-who mode, enable only one option from following 5:
# Use libvirt to list virtual guests [default]
#VIRTWHO_LIBVIRT=1
# Use vdsm to list virtual guests
#VIRTWHO_VDSM=0
# Register ESX machines using vCenter
#VIRTWHO_ESX=0
# Register guests using RHEV-M
#VIRTWHO_RHEVM=0
# Register guest using Hyper-V
#VIRTWHO_HYPERV=0

# Options for Libvirt mode
#VIRTWHO_LIBVIRT_OWNER=
#VIRTWHO_LIBVIRT_ENV=
#VIRTWHO_LIBVIRT_SERVER=
#VIRTWHO_LIBVIRT_USERNAME=
#VIRTWHO_LIBVIRT_PASSWORD=

# Options for ESX mode
#VIRTWHO_ESX_OWNER=
#VIRTWHO_ESX_ENV=
#VIRTWHO_ESX_SERVER=
#VIRTWHO_ESX_USERNAME=
#VIRTWHO_ESX_PASSWORD=

# Options for RHEV-M mode
#VIRTWHO_RHEVM_OWNER=
#VIRTWHO_RHEVM_ENV=
#VIRTWHO_RHEVM_SERVER=
#VIRTWHO_RHEVM_USERNAME=
#VIRTWHO_RHEVM_PASSWORD=

# Options for HYPER-V mode
#VIRTWHO_HYPERV_OWNER=
#VIRTWHO_HYPERV_ENV=
#VIRTWHO_HYPERV_SERVER=
#VIRTWHO_HYPERV_USERNAME=
#VIRTWHO_HYPERV_PASSWORD=

# Options for Satellite 5 backend
#VIRTWHO_SATELLITE_SERVER=
#VIRTWHO_SATELLITE_USERNAME=
#VIRTWHO_SATELLITE_PASSWORD=

virt-who.d/qua-esx

[qua-esx]
type=esx
server=vsphere.example.com
username=virt-who-user
encrypted_password=44cbc12506bb70768605e099d082ad95e2a1813b90d63b7f0bdf056e5e9ce2b7
owner=default
#env=default-Nonprod
env=Library
#rhsm_hostname=127.0.0.1
rhsm_hostname=satellite.example.com
rhsm_username=admin
#rhsm_encrypted_password=30792e07e05cb8b922bb9f1a0bb3231c1b7b1e0ddcca9ffa13a7a81dd47c90cb
rhsm_password=originalsatellitepassowrd
rhsm_prefix=/rhsm
hypervisor_id=hostname
http_proxy=http://svc_proxy:example.example.com:8080

Comment 3 Radek Novacek 2016-06-21 06:59:43 UTC
It seems to be problem in the Satellite itself, as it replies with error code 500:


java.lang.NullPointerException
4821         org.candlepin.common.exceptions.mappers.RollbackExceptionMapper.toResponse(RollbackExceptionMapper.java:55)
4822         org.candlepin.common.exceptions.mappers.RollbackExceptionMapper.toResponse(RollbackExceptionMapper.java:36)

Comment 4 Michael Stead 2016-06-23 14:52:00 UTC
Any chance of getting a copy of the candlepin logs after reproducing with DEBUG logging turned on?

Comment 5 Michael Stead 2016-06-23 14:53:45 UTC
Debug logging in candlepin that is.

You can do this by updating the candlepin config file and restarting sat6 services.

log4j.logger.org.candlepin=DEBUG

Comment 6 Michael Stead 2016-06-23 15:03:11 UTC
Would also help to get the log of the host/guest mapping that is sent from virt-who to candlepin.

Comment 7 Michael Stead 2016-06-23 18:11:16 UTC
This is happening because candlepin

Looking at the logs in the customer case,

Comment 8 Michael Stead 2016-06-23 18:16:06 UTC
This is happening because a guest id of "" is being reported to candlepin, which has a not-null constrain on that field.

Looking at the logs in the customer's sos report, you can see that when virt-who checks in with candlepin, it is passing GuestId with an empty guestId attribute.


        {
            "guestId": "", 
            "state": 5, 
            "attributes": {
                "active": 0, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }


virt-who should not be reporting these guests. Candlepin should also reject any reported guest with a null/empty guestId attibute.

Comment 9 Michael Stead 2016-06-23 19:48:38 UTC
This has been fixed in virt-who [1] and will be available in the next released version.

[1] https://github.com/virt-who/virt-who/commit/208b7d5afa8ef12b6b0fabe5e9d5e28b0f638f1d


The only workaround right now that wouldn't require updating either candlepin or virt-who, would be to ensure that esx does not have any guests without a guest id. This would ensure that virt-who does not report them to candlepin.

Comment 10 Michael Stead 2016-06-24 18:31:05 UTC
This was already fixed in candlepin-0.9.49.13+

Updating the candlepin version on the satellite should resolve this problem.

Comment 11 Michael Stead 2016-06-24 18:53:35 UTC
Fixed in candlepin-0.9.49.13+

Comment 13 Eko 2016-08-10 05:21:52 UTC
Check virt-who-0.17-7.el7.noarch package, the commit[1] is not built in virt-who package
[1]. https://github.com/virt-who/virt-who/commit/208b7d5afa8ef12b6b0fabe5e9d5e28b0f638f1d

I have two question:
 1). How to get a empty guestId value? and why is the guestId value missing?
 2). Radek, could you need to merge the above commit to fix this issue?

Comment 14 Radek Novacek 2016-08-11 14:27:25 UTC
Eko, I've merged that commit to virt-who-0.17-8.el7.

Comment 15 jcallaha 2016-08-11 16:37:46 UTC
Radek,

can you answer Eko's first question?

Comment 16 jcallaha 2016-08-12 21:33:07 UTC
I am currently unable to get virt-who to report vm's with a blank guestId.

...
        {
            "guestId": "422e760d-94ff-5e5a-e75d-41507db21a73", 
            "state": 5, 
            "attributes": {
                "active": 0, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }, 
        {
            "guestId": "422e9fc7-3929-cac0-b67b-2cca66ac8a5d", 
            "state": 5, 
            "attributes": {
                "active": 0, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }, 
        {
            "guestId": "422e04e8-164a-bb7e-7b60-633c3ea63ab7", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }, 
...

Comment 17 jcallaha 2016-08-13 21:04:46 UTC
Digging a bit more into this and i'm not seeing the changes in the file from the commit mentioned above. I've put the contents of the upstream file (https://github.com/virt-who/virt-who/blob/208b7d5afa8ef12b6b0fabe5e9d5e28b0f638f1d/virtwho/virt/esx/esx.py) into a local file and these are the results of a diff.

[root@rhsm-qe-1 /]# diff upstream.py /usr/lib/python2.7/site-packages/virtwho/virt/esx/esx.py
254c254,257
<                     uuid = "%(config.network.dnsConfig.hostName)s.%(config.network.dnsConfig.domainName)s" % host
---
>                     uuid = host['config.network.dnsConfig.hostName']
>                     domain_name = host['config.network.dnsConfig.domainName']
>                     if domain_name:
>                         uuid = '{0}.{1}'.format(uuid, domain_name)
271,273d273
<                     if not vm['config.uuid'].strip():
<                         self.logger.debug("Guest '%s' has empty 'config.uuid' property", vm_id.value)
<                         continue
286c286,289
<                 name = '%(config.network.dnsConfig.hostName)s.%(config.network.dnsConfig.domainName)s' % host
---
>                 name = host['config.network.dnsConfig.hostName']
>                 domain_name = host['config.network.dnsConfig.domainName']
>                 if domain_name:
>                     name = '{0}.{1}'.format(name, domain_name)


I've got the latest version available in the Sat 6.2.1 Snap 1.2 tools package.

[root@rhsm-qe-1 /]# rpm -qa | grep virt-who
virt-who-0.17-9.el7sat.noarch

Comment 18 jcallaha 2016-08-15 14:53:01 UTC
Failed QA in Satellite 6.2.1 Snap 1.2

These changes are not in the current build. see above.

Comment 20 Radek Novacek 2016-08-16 06:03:58 UTC
I'm sorry but I don't know how to reproduce empty guest UUIDs in the ESX. I didn't manage to reproduce it anyhow so I just created a blind workaround for it.

There are some version mismatch happening right now, the patch for this issue is applied in virt-who-0.17-8.el7, but not in the virt-who-0.17-9.el7sat. I will try to fix this mess and let you know.

Comment 21 Barnaby Court 2016-08-25 14:35:48 UTC
*** Bug 1359825 has been marked as a duplicate of this bug. ***

Comment 24 Mihir Lele 2017-06-08 12:44:55 UTC
Hello,

The Customer has updated the virt-who to virt-who-0.17-11.el7_3 and his candlepin is at candlepin-0.9.54.20-1.el7.noarch. Still he is facing the same error:

# virt-who --one-shot
2017-06-07 14:29:28,614 INFO: Using configuration "xyz-vcneter" ("esx" mode)
2017-06-07 14:29:28,615 INFO: Using reporter_id='uxvckv005a0412.cbp.dhs.gov-d110c144efcc447ebeed404404040404dfa'
2017-06-07 14:29:32,141 INFO: Sending update in hosts-to-guests mapping for config "xyz-vcneter": 11 hypervisors and 111 guests found
2017-06-07 14:29:43,834 ERROR: Response: 500
2017-06-07 14:29:43,835 ERROR: JSON parsing error: No JSON object could be decoded
2017-06-07 14:29:43,835 ERROR: Unable to send data: Communication with subscription manager failed with code 500: Server error attempting a POST to /rhsm/hypervisors?owner=test&env=Library returned status 500

Comment 32 Satellite Program 2018-02-21 16:54:17 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.
> 
> For information on the advisory, and where to find the updated files, follow the link below.
> 
> If the solution does not work for you, open a new bug report.
> 
> https://access.redhat.com/errata/RHSA-2018:0336