Bug 1247866 - virt-who can't connect to vcenter instantly after reboot vcenter
virt-who can't connect to vcenter instantly after reboot vcenter
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: virt-who (Show other bugs)
7.2
x86_64 Linux
unspecified Severity medium
: rc
: ---
Assigned To: Radek Novacek
Eko
: Reopened
Depends On: 1231602
Blocks:
  Show dependency treegraph
 
Reported: 2015-07-29 02:21 EDT by Liushihui
Modified: 2016-11-30 19:36 EST (History)
7 users (show)

See Also:
Fixed In Version: virt-who-0.17-1.el7
Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: 1231602
Environment:
Last Closed: 2016-11-04 01:05:51 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 1 Eko 2015-09-01 04:11:25 EDT
1. start reboot vcenter and virt-who stop to create message in rhsm.log
============The Time is 15:52:22 ==============
2015-09-01 15:52:22,880 [INFO]  @subscriptionmanager.py:165 - Sending update in hosts-to-guests mapping:



2. vcenter is restarting now, virt-who will create many timeout message
============The Time is 15:52:22 ==============
2015-09-01 15:54:22,366 [DEBUG]  @esx.py:105 - Wait for ESX event finished, timeout

3. after restart, the windows Server Manager is still not ready, but virt-who will feedback error message as following:
============The Time is 15:54:38 ==============
2015-09-01 15:54:38,861 [DEBUG]  @esx.py:55 - Log into ESX
2015-09-01 15:54:41,137 [ERROR]  @virt.py:304 - Virt backend 'env/cmdline' fails with exception:
Traceback (most recent call last):
  File "/usr/share/virt-who/virt/virt.py", line 300, in run
    self._run()
  File "/usr/share/virt-who/virt/esx/esx.py", line 68, in _run
    self._prepare()
  File "/usr/share/virt-who/virt/esx/esx.py", line 56, in _prepare
    self.login()
  File "/usr/share/virt-who/virt/esx/esx.py", line 229, in login
    self.sc = self.client.service.RetrieveServiceContent(_this=self.moRef)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 542, in __call__
    return client.invoke(args, kwargs)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 602, in invoke
    result = self.send(soapenv)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 653, in send
    result = self.failed(binding, e)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 714, in failed
    raise Exception((status, reason))
Exception: (503, u'Service Unavailable (Failed to connect to endpoint: [class Vmacore::Http::LocalServiceSpec:0000000000EA7300] _serverNamespace = /sdk _isRedirect = false _port = 8085)')
2015-09-01 15:54:41,137 [INFO]  @virt.py:313 - Waiting 5 seconds before retrying backend 'env/cmdline'

4. waiting for a moment, Server Manager is ready, virt-who can feedback the right mapping info
============The Time is 15:59:50 ==============
2015-09-01 15:59:50,564 [DEBUG]  @esx.py:55 - Log into ESX
2015-09-01 15:59:51,691 [DEBUG]  @esx.py:58 - Creating ESX event filter
2015-09-01 15:59:56,559 [DEBUG]  @esx.py:142 - Waiting for ESX changes
2015-09-01 15:59:56,561 [DEBUG]  @subscriptionmanager.py:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2015-09-01 15:59:58,156 [DEBUG]  @subscriptionmanager.py:146 - Checking if server has capability 'hypervisor_async'
2015-09-01 16:00:00,205 [DEBUG]  @subscriptionmanager.py:158 - Server does not have 'hypervisors_async' capability
2015-09-01 16:00:00,206 [INFO]  @subscriptionmanager.py:165 - Sending update in hosts-to-guests mapping: {
    "aee4ff00-8c33-11e2-994a-6c3be51d959a": [
        {
            "guestId": "564d3964-c083-deba-55f9-000279dd82f9", 
            "state": 5, 
            "attributes": {
                "active": 0, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }
    ]
}



In general, there is no long delay after reboot the vcenter host, but I don't know the error message of step(3) is expected or not
Comment 2 Radek Novacek 2015-09-01 04:17:51 EDT
It seems that the error in step 3 is some kind of internal failure in vCenter (some internal service is not yet started after the reboot).

I don't think there is much that virt-who can do about it, apart for hiding the error (that would be bad idea because it can hide other problems).

It looks like the bug is now fixed. Can we close this report now?
Comment 3 Liushihui 2015-09-01 22:30:04 EDT
Radek, According to the comment 1,this problem has been resolved in the latest virt-who. I think we can close this bug on this version. Therefore, close it as worksforme. Thanks.

Checked version:
virt-who-0.14-4.el7.noarch
subscription-manager-1.15.9-6.el7.x86_64
python-rhsm-1.15.4-2.el7.x86_64
Satellite6.1.0-20150820.0
Comment 4 Liushihui 2015-09-16 05:48:59 EDT
It has reproduced on virt-who-0.14-7.el7.noarch, Therefore, reopen it.

Reproduced version:
virt-who-0.14-7.el7.noarch
subscription-manager-1.15.9-7.el7.x86_64
python-rhsm-1.15.4-4.el7.x86_64
SAM-1.4.1-RHEL-6-20141113.0

Reproduced process:
1. Register system to SAM
2. Configure virt-who run at esx mode, and the refresh interval is 5s.
3. Restart virt-who service, virt-who send the host/guest mapping to SAM
4. Stop vcenter, then start vcenter, check the virt-who's log

Result:
virt-who can't connect to vcenter until 30 min later.

2015-09-16 17:13:22,820 [DEBUG]  @esx.py:142 - Waiting for ESX changes
2015-09-16 17:13:22,824 [DEBUG]  @subscriptionmanager.py:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2015-09-16 17:13:22,846 [DEBUG]  @subscriptionmanager.py:146 - Checking if server has capability 'hypervisor_async'
2015-09-16 17:13:22,867 [DEBUG]  @subscriptionmanager.py:158 - Server does not have 'hypervisors_async' capability
2015-09-16 17:13:22,868 [INFO]  @subscriptionmanager.py:165 - Sending update in hosts-to-guests mapping: {
    "93b45c80-28ec-11e2-87c4-b4b52fe0f08d": [
        {
            "guestId": "422c92cb-14b2-e7a5-77fe-a530c7311de7", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }
    ], 
    "826c4f80-28ee-11e2-b149-b4b52fe0ce88": []
}

******************After 30min, virt-who send h/g mapping to SAM again********

2015-09-16 17:43:22,927 [DEBUG]  @esx.py:105 - Wait for ESX event finished, timeout
2015-09-16 17:43:22,968 [DEBUG]  @esx.py:55 - Log into ESX
2015-09-16 17:43:23,352 [DEBUG]  @esx.py:58 - Creating ESX event filter
2015-09-16 17:43:23,410 [DEBUG]  @esx.py:142 - Waiting for ESX changes
2015-09-16 17:43:23,412 [DEBUG]  @subscriptionmanager.py:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2015-09-16 17:43:23,443 [DEBUG]  @subscriptionmanager.py:146 - Checking if server has capability 'hypervisor_async'
2015-09-16 17:43:23,473 [DEBUG]  @subscriptionmanager.py:158 - Server does not have 'hypervisors_async' capability
2015-09-16 17:43:23,473 [INFO]  @subscriptionmanager.py:165 - Sending update in hosts-to-guests mapping: {
    "93b45c80-28ec-11e2-87c4-b4b52fe0f08d": [
        {
            "guestId": "422c92cb-14b2-e7a5-77fe-a530c7311de7", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "esx", 
                "hypervisorType": "vmware"
            }
        }
    ], 
    "826c4f80-28ee-11e2-b149-b4b52fe0ce88": []
}


Expected result:
virt-who should report host/guest to SAM immediate after reboot vcenter
Comment 5 Radek Novacek 2015-09-22 02:58:25 EDT
This bug seems to be similar to bug 1247866. Moving to RHEL-7.3.
Comment 7 Radek Novacek 2016-05-03 09:39:22 EDT
Since this bug is cloned from bug #1231602 that is already fixed in RHEL-6.8 and the code base for 7.3 rebase is the same, the upcoming rebase of virt-who should fix this issue.
Comment 8 Radek Novacek 2016-05-17 09:00:17 EDT
Fixed in virt-who-0.17-1.el7.
Comment 10 Liushihui 2016-06-02 01:53:19 EDT
Verified it on virt-who-0.17-2.el7.noarch since virt-who can connect vcenter every refresh interval, virt-who also can send  h/g mapping info to server immediately after reboot vcenter successfully. Therefore, verify it.
Comment 13 errata-xmlrpc 2016-11-04 01:05:51 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2387.html

Note You need to log in before you can comment on or make changes to this bug.