+++ This bug was initially created as a clone of Bug #1380510 +++

+++ This bug was initially created as a clone of Bug #1362724 +++

Description of problem:
If there was a hostname exist on server, virt-who send the same hostname to server again, although it won't generate two same hostnames in server any more, virt-who will updated the previous host's uuid and it result in virt-who can't send mapping info to server.

Version-Release number of selected component (if applicable):
virt-who-0.17-7.el7.noarch
subscription-manager-1.17.9-1.el7.x86_64
python-rhsm-1.17.5-1.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
Precondition:
Virt-who work at a RHEL host and this host has been added to rhevm

Process:
1. Register RHEL Host to satellite6.2, the registed host's uuid is "38ceb55a-e12d-443d-8f42-e62be5d75010", please see screenshot in attachment old_uuid.jpeg
[root@hp-dl560egen8-01 virt-who.d]# subscription-manager  register --username=admin --password=admin
Registering to: hp-dl2x170g6-01.rhts.eng.bos.redhat.com:443/rhsm
The system has been registered with ID: 38ceb55a-e12d-443d-8f42-e62be5d75010 

2. Configure virt-who work at rhevm mode and report hostname instead of hostuuid
[root@hp-dl560egen8-01 virt-who.d]# cat /etc/virt-who.d/rhevm 
[test-rhevm1]
type=rhevm
server=https://ibm-x3690x5-01.rhts.eng.bos.redhat.com:443/ovirt-engine/
username=admin@internal
encrypted_password=48aeec0241dc902f383f94f32efc2fb3
owner=ACME_Corporation
env=Library
hypervisor_id=hostname

3. Restart virt-who and check virt-who's log, virt-who send host/guest mapping info to satellite sucessfully and virt-who updated the host's uuid to "983d6840-e219-4521-9a11-fce9ba1e8f68". see screenshot in attachment updated_uuid.jpeg
[root@hp-dl560egen8-01 ~]# tail -f /var/log/rhsm/rhsm.log
2016-08-02 04:35:38,958 [virtwho.init DEBUG] MainProcess(3990):MainThread @executor.py:__init__:65 - Using config named 'test-rhevm1'
2016-08-02 04:35:38,959 [virtwho.init INFO] MainProcess(3990):MainThread @main.py:main:160 - Using configuration "test-rhevm1" ("rhevm" mode)
2016-08-02 04:35:38,959 [virtwho.init INFO] MainProcess(3990):MainThread @main.py:main:162 - Using reporter_id='hp-dl560egen8-01.khw.lab.eng.bos.redhat.com-802e3ee1794e4d728db29e3f9fbae108'
2016-08-02 04:35:38,962 [virtwho.main DEBUG] MainProcess(3990):MainThread @executor.py:run:171 - Starting infinite loop with 60 seconds interval
2016-08-02 04:35:39,006 [virtwho.test-rhevm1 DEBUG] RhevM-1(3997):MainThread @virt.py:run:364 - Virt backend 'test-rhevm1' started
2016-08-02 04:35:39,540 [virtwho.test-rhevm1 DEBUG] RhevM-1(3997):MainThread @virt.py:enqueue:357 - Report for config "test-rhevm1" gathered, putting to queue for sending
2016-08-02 04:35:39,555 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:_connect:123 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2016-08-02 04:35:39,915 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:171 - Checking if server has capability 'hypervisor_async'
2016-08-02 04:35:40,231 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:183 - Server does not have 'hypervisors_async' capability
2016-08-02 04:35:40,231 [virtwho.main INFO] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:194 - Sending update in hosts-to-guests mapping for config "test-rhevm1": 2 hypervisors and 1 guests found
2016-08-02 04:35:40,231 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:195 - Host-to-guest mapping: {
    "hp-dl560egen8-01.khw.lab.eng.bos.redhat.com": [
        {
            "guestId": "530c34dc-2086-4f1d-a509-c417950bd0cb", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "rhevm"
            }
        }
    ], 
    "dell-pe1950-06.rhts.englab.brq.redhat.com": []
}
2016-08-02 04:35:42,761 [virtwho.main DEBUG] MainProcess(3990):MainThread @executor.py:send_report:101 - Report for config "test-rhevm1" sent


4. In rhevm webUI, update host or guest's info, eg: add a guest, wait for 60s, check virt-who's log
[root@hp-dl560egen8-01 ~]# tail -f /var/log/rhsm/rhsm.log
2016-08-02 04:37:40,711 [virtwho.test-rhevm1 DEBUG] RhevM-1(3997):MainThread @virt.py:enqueue:357 - Report for config "test-rhevm1" gathered, putting to queue for sending
2016-08-02 04:37:40,715 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:_connect:123 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2016-08-02 04:37:41,168 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:171 - Checking if server has capability 'hypervisor_async'
2016-08-02 04:37:41,476 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:183 - Server does not have 'hypervisors_async' capability
2016-08-02 04:37:41,476 [virtwho.main INFO] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:194 - Sending update in hosts-to-guests mapping for config "test-rhevm1": 2 hypervisors and 2 guests found
2016-08-02 04:37:41,476 [virtwho.main DEBUG] MainProcess(3990):MainThread @subscriptionmanager.py:hypervisorCheckIn:195 - Host-to-guest mapping: {
    "hp-dl560egen8-01.khw.lab.eng.bos.redhat.com": [
        {
            "guestId": "530c34dc-2086-4f1d-a509-c417950bd0cb", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "rhevm"
            }
        }
    ], 
    "dell-pe1950-06.rhts.englab.brq.redhat.com": [
        {
            "guestId": "4c4f9cbd-d811-4061-9aeb-050b774b870a", 
            "state": 5, 
            "attributes": {
                "active": 0, 
                "virtWhoType": "rhevm"
            }
        }
    ]
}
2016-08-02 04:37:41,718 [virtwho.main ERROR] MainProcess(3990):MainThread @executor.py:send:143 - Unable to send data: Communication with subscription manager failed with code 404: Couldn't find consumer '38ceb55a-e12d-443d-8f42-e62be5d75010'
2016-08-02 04:37:41,718 [virtwho.main DEBUG] MainProcess(3990):MainThread @executor.py:send_report:108 - Report from "test-rhevm1" failed to sent

5. Check host's registered status.
[root@hp-dl560egen8-01 virt-who.d]# subscription-manager  identity
system identity: 38ceb55a-e12d-443d-8f42-e62be5d75010
name: hp-dl560egen8-01.khw.lab.eng.bos.redhat.com
org name: Default Organization
org ID: Default_Organization
Couldn't find consumer '38ceb55a-e12d-443d-8f42-e62be5d75010'

Actual results:
Failed to send update info to server since the old host's uuid(38ceb55a-e12d-443d-8f42-e62be5d75010) has been updated by virt-who.
Unable to send data: Communication with subscription manager failed with code 404: Couldn't find consumer '38ceb55a-e12d-443d-8f42-e62be5d75010'

Expected results:
Virt-who should send update info to server successfully 

Additional info:

--- Additional comment from Liushihui on 2016-08-02 21:58 EDT ---



--- Additional comment from Liushihui on 2016-08-02 21:58 EDT ---



--- Additional comment from Radek Novacek on 2016-08-09 11:30:56 EDT ---

It seems that virt-who is sending correct data. Reassigning to candlepin for further investigation.

--- Additional comment from Chris Snyder on 2016-09-30 11:45:13 EDT ---

Please investigate the use of the virt-who configuration option 'hypervisor_id=hostname'. It seems that this configuration will nearly always break virt-who's mapping on the candlepin back to the right consumer.

--- Additional comment from Daniel Messer on 2016-11-03 16:36:46 EDT ---

I can confirm the behavior reported. Initial registration of a RHEV hypervisor will result in UUID v1. Upon first detection of guests on that hypervisor by virt-who the corresponding content-host will change to UUID v2.

Subsequent subscription-manager commands on the hypervisor will fail. Also a VDC subscription attached to the hypervisor will become invalid, all guests will loose their entitlement. As such it's not possible to use virt-who with RHEV and consequently Satellite with RHEV.

--- Additional comment from Kevin Howell on 2016-12-15 11:42:02 EST ---

Daniel,

Can you please confirm whether using 'hypervisor_id=uuid' fixes the issues?

--- Additional comment from Daniel Messer on 2017-01-09 07:54:27 EST ---

Hi Kevin, I don't have access to the environment anymore but it fixes the issue - however the hypervisors will appear with UUID-based names in Satellite which is less than ideal. Also this differs from the docs.

--- Additional comment from Chris Snyder on 2017-01-12 10:41:32 EST ---

I think we should convert this to an RFE because of the following.
The configuration parameter 'hypervisor_id' allows specifying in virt-who what value to use to uniquely identify a particular hypervisor. Allowing this value to be set to something that will change and is not unique (for example, setting the parameter to 'hostname') will cause issues because those properties are expected of the data being reported by virt-who on the server (candlepin) side of hypervisor checkins.

As far as I can tell the reason that this value is configurable is because this same value is also used to generate a display name for hypervisor instances. Naturally, traditional UUIDs do not make for very human-friendly display names.

What I propose we do is investigate if it is possible to get a UUID value from all supported virtualization frameworks (HyperV, VMWare ESX/vSphere, etc). If we can, we should only use the UUID from all frameworks as the unique hypervisor_id.
We should also collect the hostname from all frameworks which support doing so and include this information with all reports from virt-who. In this way we can use the hostname as a display name on the server side while maintaining a unique way for us to refer to a particular hypervisor.

We can use this bug to track the work done server side to use the reported hostname as the display name. I'll clone this bug to track the work necessary on the virt-who side.