Bug 1271158 - Failed to delete the host or hypervisor from stage candlepin by web UI
Failed to delete the host or hypervisor from stage candlepin by web UI
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: python-rhsm (Show other bugs)
6.8
x86_64 Linux
medium Severity low
: pre-dev-freeze
: ---
Assigned To: Chris Snyder
Eko
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-13 05:28 EDT by Eko
Modified: 2016-05-10 16:37 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-10 16:37:52 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Eko 2015-10-13 05:28:41 EDT
Description of problem:
register a host to stage candlepin, and then try to delete it from the web ui,
the host is deleted successfully, but there are some error message found from rhsm.log 

Version-Release number of selected component (if applicable):
 - python-rhsm-1.15.4-5.el7.x86_64
 - subscription-manager-1.15.9-14.el7.x86_64
 - virt-who-0.14-9.el7.noarch
 - RHEL7.2-20151008.0-Server-x86_64 + vdsm mode against Stage candlepin

How reproducible:


Steps to Reproduce:

1. prepare a host and register it to stage candlepin by CLI
# subscription-manager register --username=virt-who-perf --password=redhat
Registering to: subscription.rhn.stage.redhat.com:443/subscription
The system has been registered with ID: 136aa925-62ab-40f1-9fa0-d49dae8116c8

2. restart virt-who service
# systemctl  restart virt-who

3. open the stage candlepin web site, you will find the host, and delete it from web ui. The unit(s) you selected have been deleted.

4. check rhsm.log, the error message as following:
2015-10-13 17:19:25,616 [INFO]  @subscriptionmanager.py:136 - Sending domain info: [
    {
        "guestId": "0bff30b0-7c7d-4884-a8a2-6b3b43daa963", 
        "state": 1, 
        "attributes": {
            "active": 1, 
            "virtWhoType": "vdsm", 
            "hypervisorType": "qemu"
        }
    }
]
2015-10-13 17:19:30,360 [ERROR]  @virtwho.py:190 - Error in communication with subscription manager:
Traceback (most recent call last):
  File "/usr/share/virt-who/virtwho.py", line 171, in send
    self._sendGuestList(report)
  File "/usr/share/virt-who/virtwho.py", line 196, in _sendGuestList
    manager.sendVirtGuests(report.guests)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 139, in sendVirtGuests
    self.connection.updateConsumer(self.uuid(), guest_uuids=serialized_guests)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 947, in updateConsumer
    ret = self.conn.request_put(method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 653, in request_put
    return self._request("PUT", method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 597, in validateResponse
    parsed['displayMessage'], parsed['deletedId'])
GoneException: Unit 136aa925-62ab-40f1-9fa0-d49dae8116c8 has been deleted

5. run "subscription-manager identity" to check, the unit is deleted.
# subscription-manager identity 
Unit 136aa925-62ab-40f1-9fa0-d49dae8116c8 has been deleted

6. but register the host again, shows "This system is already registered"
# subscription-manager register --username=virt-who-perf --password=redhat
This system is already registered. Use --force to override

7. need to run clean command and then register the host normally
# subscription-manager clean
All local data removed

# subscription-manager register --username=virt-who-perf --password=redhat
Registering to: subscription.rhn.stage.redhat.com:443/subscription
The system has been registered with ID: d074447e-917b-4e2f-9268-f36864fe25fb


Actual results:
if delete the host by stage candlepin web ui, the rhsm.log will show error message as "GoneException: Unit 136aa925-62ab-40f1-9fa0-d49dae8116c8 has been deleted".
but the host is not deleted normally, need to run "subscription-manager clean"

Expected results:
if delete the host by stage candlepin web ui, the rhsm.log should show "SubscriptionManagerUnregisteredError: Unable to read certificate, system is not registered or you are not root", don't need to clean for register the host again.

Additional info:
Comment 2 Radek Novacek 2015-10-13 06:23:16 EDT
It doesn't look like a bug in virt-who. It seems that either candlepin fails to delete the host properly or python-rhsm doesn't understand the reply from candlepin.

I'll move it to candlepin for evaluation.
Comment 3 Barnaby Court 2015-11-20 10:35:51 EST
This is a doc issue that the conneciton module in python-rhsm does not document what exceptions it can raise. After that is doc'd there is a secondary bug that virt-who is not handling the exception.
Comment 5 Chris Snyder 2015-12-16 17:12:47 EST
As of the following commits it seems virt-who now properly handles GoneException raised by python-rhsm:
"""
commit eb66da3d8f89307835dcb1cfbd5a715e4487d19e
Author: Radek Novacek <rnovacek@redhat.com>
Date:   Tue Nov 24 15:24:26 2015 +0100

    subscriptionmanager: handle GoneException correctly
    
    Just logging the error message is not correct, virt-who would try
    to create new job, but no job was created server side. Better to raise
    an exception.

commit 88f8f4e2f4dd92615d7922e4282da038491a5727
Author: Radek Novacek <rnovacek@redhat.com>
Date:   Tue Nov 24 15:21:09 2015 +0100

    subscriptionmanager: handle GoneException when consumer is deleted
"""

The solution to this BZ will be updated python-rhsm documentation regarding what can be raised by the methods used here by virt-who.
Comment 6 Chris Snyder 2015-12-17 15:55:16 EST
PR here: https://github.com/candlepin/python-rhsm/pull/161
Comment 7 Chris Snyder 2016-01-04 14:22:48 EST
The PR above with the docs fixes has been merged to upstream master. As we have not branched yet for rhel6.8 these changes will make the next build of python-rhsm.
Comment 9 Shwetha Kallesh 2016-01-20 06:31:30 EST
An error message " Unable to generate a new identity for the system: Unit 6444637d-12ec-4e15-9b87-76456cf748e9 has been deleted" is logged into the /var/log/rhsm/rhsm.log now if you try to register again on a host that is  deleted from stage candlepin web UI

server type: Red Hat Subscription Management
subscription management server: 0.9.51.14-1
subscription management rules: 5.15.1
subscription-manager: 1.16.8-1.el6
python-rhsm: 1.16.6-1.el6


[root@hp-dl320egen8-01 ~]# subscription-manager register --username=virt-who-perf --password=redhat
Registering to: subscription.rhn.stage.redhat.com:443/subscription
The system has been registered with ID: 6444637d-12ec-4e15-9b87-76456cf748e9
 
[root@hp-dl320egen8-01 ~]# service virt-who restart
Stopping virt-who: [  OK  ]
Starting virt-who: [  OK  ]
 
[root@hp-dl320egen8-01 ~]# subscription-manager identity
Unit 6444637d-12ec-4e15-9b87-76456cf748e9 has been deleted
[root@hp-dl320egen8-01 ~]# subscription-manager register --username=virt-who-perf --password=redhat
This system is already registered. Use --force to override
 
[root@hp-dl320egen8-01 ~]# tail -f /var/log/rhsm/rhsm.log
        "state": 1,
        "attributes": {
            "active": 1,
            "hypervisorVersion": "0.12.1",
            "virtWhoType": "libvirt",
            "hypervisorType": "QEMU"
        }
    }
]
2016-01-19 08:45:08,630 [INFO]  @virtwho.py:250 - virt-who guest list update successful
2016-01-19 08:47:53,074 [DEBUG] subscription-manager:38457 @ga_loader.py:89 - ga_loader GaImporterGtk2
2016-01-19 08:47:53,077 [DEBUG] subscription-manager:38457 @plugins.py:569 - loaded plugin modules: [<module 'container_content' from '/usr/share/rhsm-plugins/container_content.pyc'>]
2016-01-19 08:47:53,077 [DEBUG] subscription-manager:38457 @plugins.py:570 - loaded plugins: {'container_content.ContainerContentPlugin': <container_content.ContainerContentPlugin object at 0x20f2790>}
2016-01-19 08:47:53,077 [DEBUG] subscription-manager:38457 @identity.py:131 - Loading consumer info from identity certificates.
2016-01-19 08:47:53,106 [INFO] subscription-manager:38457 @managercli.py:367 - Client Versions: {'python-rhsm': '1.16.5-1.el6', 'subscription-manager': '1.16.7-1.el6'}
2016-01-19 08:47:53,106 [INFO] subscription-manager:38457 @connection.py:815 - Connection built: host=subscription.rhn.stage.redhat.com port=443 handler=/subscription auth=identity_cert ca_dir=/etc/rhsm/ca/ verify=False
2016-01-19 08:47:53,106 [INFO] subscription-manager:38457 @connection.py:815 - Connection built: host=subscription.rhn.stage.redhat.com port=443 handler=/subscription auth=none
2016-01-19 08:47:53,116 [INFO] subscription-manager:38457 @managercli.py:342 - Consumer Identity name=hp-dl320egen8-01.rhts.eng.bos.redhat.com uuid=6444637d-12ec-4e15-9b87-76456cf748e9
2016-01-19 08:47:53,117 [DEBUG] subscription-manager:38457 @connection.py:509 - Loaded CA certificates from /etc/rhsm/ca/: redhat-uep.pem, redhat-entitlement-authority.pem
2016-01-19 08:47:53,117 [DEBUG] subscription-manager:38457 @connection.py:557 - Making request: GET /subscription/consumers/6444637d-12ec-4e15-9b87-76456cf748e9/owner
2016-01-19 08:47:53,663 [DEBUG] subscription-manager:38457 @connection.py:589 - Response: status=410
2016-01-19 08:47:53,664 [ERROR] subscription-manager:38457 @managercli.py:704 - Unit 6444637d-12ec-4e15-9b87-76456cf748e9 has been deleted
Traceback (most recent call last):
  File "/usr/share/rhsm/subscription_manager/managercli.py", line 671, in _do_command
    owner = self.cp.getOwner(consumerid)
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 1083, in getOwner
    return self.conn.request_get(method)
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 681, in request_get
    return self._request("GET", method)
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 598, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 634, in validateResponse
    parsed['deletedId'])
GoneException: Unit 6444637d-12ec-4e15-9b87-76456cf748e9 has been deleted
2016-01-19 08:47:53,665 [ERROR] subscription-manager:38457 @managercli.py:705 - Error: Unable to generate a new identity for the system: Unit 6444637d-12ec-4e15-9b87-76456cf748e9 has been deleted
2016-01-19 08:59:41,650 [DEBUG] subscription-manager:39210 @ga_loader.py:89 - ga_loader GaImporterGtk2
2016-01-19 08:59:41,654 [DEBUG] subscription-manager:39210 @plugins.py:569 - loaded plugin modules: [<module 'container_content' from '/usr/share/rhsm-plugins/container_content.pyc'>]
2016-01-19 08:59:41,654 [DEBUG] subscription-manager:39210 @plugins.py:570 - loaded plugins: {'container_content.ContainerContentPlugin': <container_content.ContainerContentPlugin object at 0x1b8b810>}
2016-01-19 08:59:41,654 [DEBUG] subscription-manager:39210 @identity.py:131 - Loading consumer info from identity certificates.
2016-01-19 08:59:41,682 [INFO] subscription-manager:39210 @managercli.py:367 - Client Versions: {'python-rhsm': '1.16.5-1.el6', 'subscription-manager': '1.16.7-1.el6'}
2016-01-19 08:59:41,683 [INFO] subscription-manager:39210 @connection.py:815 - Connection built: host=subscription.rhn.stage.redhat.com port=443 handler=/subscription auth=identity_cert ca_dir=/etc/rhsm/ca/ verify=False
2016-01-19 08:59:41,683 [INFO] subscription-manager:39210 @connection.py:815 - Connection built: host=subscription.rhn.stage.redhat.com port=443 handler=/subscription auth=none
2016-01-19 08:59:41,684 [INFO] subscription-manager:39210 @managercli.py:367 - Client Versions: {'python-rhsm': '1.16.5-1.el6', 'subscription-manager': '1.16.7-1.el6'}
2016-01-19 08:59:41,693 [INFO] subscription-manager:39210 @managercli.py:342 - Consumer Identity name=hp-dl320egen8-01.rhts.eng.bos.redhat.com uuid=6444637d-12ec-4e15-9b87-76456cf748e9
Comment 11 errata-xmlrpc 2016-05-10 16:37:52 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0797.html

Note You need to log in before you can comment on or make changes to this bug.