Bug 1159229
| Summary: | RFE: add support for parallel reading of data from backends | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Liushihui <shihliu> |
| Component: | virt-who | Assignee: | Radek Novacek <rnovacek> |
| Status: | CLOSED ERRATA | QA Contact: | Li Bin Liu <liliu> |
| Severity: | high | Docs Contact: | Jiri Herrmann <jherrman> |
| Priority: | unspecified | ||
| Version: | 7.1 | CC: | gxing, ovasik, sgao |
| Target Milestone: | rc | Keywords: | FutureFeature |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | virt-who-0.14-1.el7 | Doc Type: | Release Note |
| Doc Text: |
Acquiring hypervisor data in parallel
With this update, virt-who is able to acquire data from multiple hypervisors in parallel. Previously, virt-who could read data only from a single hypervisor at a time, and if one hypervisor in a series was nonfunctional, virt-who waited for its response and thus failed. Reading parallel hypervisors works around this problem and prevents the described failure.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-11-19 11:55:46 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This is expected behaviour, virt-who is now working synchronously, checking one server after another. When obtaining data from one server takes long time, it will delay the others. It might be possible to do this in parallel. But it will be quite big change that I'm not comfortable to put to 7.1. I'll transform this bug to RFE for 7.1 to add parallel checking support to virt-who. This feature has been implemented upstream and will be fixed by virt-who rebase (bug #1200374). Fixed in virt-who-0.14-1.el7. verified it on virt-who-0.14-1.el7.noarch since virt-who can support parallel checking. Therefore, verify it. Verified version: virt-who-0.14-1.el7.noarch subscription-manager-1.15.5-1.el7.x86_64 python-rhsm-1.15.3-1.el7.x86_64 Verified process: 1. Register system to Satellite server [root@hp-z220-05 ~]# subscription-manager register --username=admin --password=admin The system has been registered with ID: 29c68cd4-7a82-45e2-85c6-e08d25e8ef86 2. Configure virt-who under the /etc/sysconfig/virt-who, make sure HYPERV_SERVER=10.66.128.26 is reacheable. # cat /etc/sysconfig/virt-who VIRTWHO_INTERVAL=10 VIRTWHO_HYPERV=1 VIRTWHO_HYPERV_OWNER=ACME_Corporation VIRTWHO_HYPERV_ENV=Library VIRTWHO_HYPERV_SERVER=10.66.128.26 VIRTWHO_HYPERV_USERNAME=administrator VIRTWHO_HYPERV_PASSWORD=qwer1234P 3. Configure virt-who under /etc/virt-who.d/virt-who as the following, server=10.66.79.83 is unreacheable. [test-rhevm1] type=rhevm server=https://10.66.79.83:443 username=admin@internal password=redhat owner=ACME_Corporation env=Library 4. Restart virt-who service, check the virt-who's log: [root@hp-z220-05 etc]# service virt-who restart Redirecting to /bin/systemctl restart virt-who.service [root@hp-z220-05 etc]# tail -f /var/log/rhsm/rhsm.log 2015-07-16 11:33:37,427 [DEBUG] @virtwho.py:91 - Using config named 'test-rhevm1' 2015-07-16 11:33:37,428 [INFO] @virtwho.py:572 - Using configuration "test-rhevm1" ("rhevm" mode) 2015-07-16 11:33:37,428 [INFO] @virtwho.py:572 - Using configuration "env/cmdline" ("hyperv" mode) 2015-07-16 11:33:37,440 [DEBUG] @virtwho.py:152 - Starting infinite loop with 3600 seconds interval 2015-07-16 11:33:37,476 [DEBUG] @hyperv.py:305 - Hyper-V url: http://10.66.128.26:5985/wsman 2015-07-16 11:33:37,483 [DEBUG] @hyperv.py:355 - Using NTLM authentication 2015-07-16 11:33:37,500 [DEBUG] @hyperv.py:383 - NTLM authentication successful 2015-07-16 11:33:37,509 [DEBUG] @hyperv.py:428 - Error when enumerating using root/virtualization namespace, trying root/virtualization/v2 namespace 2015-07-16 11:33:37,511 [DEBUG] @hyperv.py:355 - Using NTLM authentication 2015-07-16 11:33:37,525 [DEBUG] @hyperv.py:383 - NTLM authentication successful 2015-07-16 11:33:37,612 [INFO] @subscriptionmanager.py:146 - Sending update in hosts-to-guests mapping: { "60527517-6284-7593-6AAB-75BF2A6375EF": [ { "attributes": { "active": 0, "virtWhoType": "hyperv" }, "guestId": "E2EC00E1-5022-2341-93FF-CD68A7BF0324", "state": 5 }, { "attributes": { "active": 0, "virtWhoType": "hyperv" }, "guestId": "FA33C6C5-3008-A942-AA59-B801093361FA", "state": 5 }, { "attributes": { "active": 0, "virtWhoType": "hyperv" }, "guestId": "10D19CA6-C63A-2F41-B9CC-9DDAC302AEDC", "state": 0 } ] } 2015-07-16 11:33:37,612 [DEBUG] @subscriptionmanager.py:111 - Authenticating with certificate: /etc/pki/consumer/cert.pem 2015-07-16 11:33:45,976 [INFO] @virtwho.py:147 - virt-who host/guest association update successful 2015-07-16 11:35:44,708 [ERROR] @virt.py:261 - Virt backend 'test-rhevm1' fails with exception: Traceback (most recent call last): File "/usr/share/virt-who/virt/virt.py", line 257, in run self._run() File "/usr/share/virt-who/virt/virt.py", line 285, in _run report = self._get_report() File "/usr/share/virt-who/virt/virt.py", line 231, in _get_report return HostGuestAssociationReport(self.config, self.getHostGuestMapping()) File "/usr/share/virt-who/virt/rhevm/rhevm.py", line 103, in getHostGuestMapping clusters_xml = ElementTree.parse(self.get(self.clusters_url)) File "/usr/share/virt-who/virt/rhevm/rhevm.py", line 89, in get return urllib2.urlopen(request) File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/usr/lib64/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 449, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1258, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open raise URLError(err) URLError: <urlopen error [Errno 110] Connection timed out> Result: virt-who can send the host/guest info immediately when it's reachable. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2370.html |
Description of problem: When vcenter which configured in the /etc/virt-who.d/virt-who can't reacheable, Although the other vcenter which configured under the sysconfig run normally, it still take a long time to send the host/guest associate of the reachable one to SAM server as virt-who need to wait for the host/guest associate of the unreacheable one at first. Version-Release number of selected component (if applicable): subscription-manager-1.13.6-1.el7.x86_64 python-rhsm-1.13.6-1.el7.x86_64 virt-who-0.11-2.el7.noarch How reproducible: Always Steps to Reproduce: 1. Register system to sam server [root@hp-z220-07 etc]# subscription-manager register --username=admin --password=admin The system has been registered with ID: 6be212f5-10ae-4d1e-9a21-8128db4096eb 2. Configure virt-who under the /etc/sysconfig/virt-who, make sure ESX_SERVER=10.66.79.89 is reacheable. # cat /etc/sysconfig/virt-who VIRTWHO_INTERVAL=10 VIRTWHO_ESX=1 VIRTWHO_ESX_OWNER=ACME_Corporation VIRTWHO_ESX_ENV=Library VIRTWHO_ESX_SERVER=10.66.79.89 VIRTWHO_ESX_USERNAME=Administrator VIRTWHO_ESX_PASSWORD=qwer1234P! 3. Configure virt-who under /etc/virt-who.d/virt-who as the following, server=10.66.79.67 is unreacheable. [test-esx1] type=esx server=10.66.79.67 username=Administrator password=qwer1234P! owner=ACME_Corporation env=Library 4. Restart virt-who service, check the virt-who's log: [root@hp-z220-07 etc]# service virt-who restart Redirecting to /bin/systemctl restart virt-who.service [root@hp-z220-07 etc]# tail -f /var/log/rhsm/rhsm.log Actual results: virt-who can't send host/guest associate of the reachable vcenter to SAM until it try to connect to the unreachable vcenter two times. please see the detail as the following: 2014-10-31 15:50:09,031 [DEBUG] @virtwho.py:83 - Using config named 'test-esx1' 2014-10-31 15:50:09,032 [INFO] @virtwho.py:457 - Using virt-who configuration: test-esx1 2014-10-31 15:50:09,032 [INFO] @virtwho.py:457 - Using virt-who configuration: virt-who 2014-10-31 15:50:09,032 [DEBUG] @virtwho.py:170 - Starting infinite loop with 10 seconds interval 2014-10-31 15:51:39,160 [ERROR] @esx.py:138 - Unable to connect to ESX Traceback (most recent call last): File "/usr/share/virt-who/virt/esx/esx.py", line 136, in scan self.client = suds.client.Client("%s/sdk/vimService.wsdl" % self.url) File "/usr/lib/python2.7/site-packages/suds/client.py", line 112, in __init__ self.wsdl = reader.open(url) File "/usr/lib/python2.7/site-packages/suds/reader.py", line 152, in open d = self.fn(url, self.options) File "/usr/lib/python2.7/site-packages/suds/wsdl.py", line 136, in __init__ d = reader.open(url) File "/usr/lib/python2.7/site-packages/suds/reader.py", line 79, in open d = self.download(url) File "/usr/lib/python2.7/site-packages/suds/reader.py", line 95, in download fp = self.options.transport.open(Request(url)) File "/usr/lib/python2.7/site-packages/suds/transport/https.py", line 60, in open return HttpTransport.open(self, request) File "/usr/lib/python2.7/site-packages/suds/transport/http.py", line 62, in open return self.u2open(u2request) File "/usr/lib/python2.7/site-packages/suds/transport/http.py", line 118, in u2open return url.open(u2request, timeout=tm) File "/usr/lib64/python2.7/urllib2.py", line 404, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 422, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1224, in https_open return self.do_open(httplib.HTTPSConnection, req) File "/usr/lib64/python2.7/urllib2.py", line 1186, in do_open raise URLError(err) URLError: <urlopen error timed out> 2014-10-31 15:51:39,161 [ERROR] @virtwho.py:118 - Error in communication with virtualization backend, trying to recover: Traceback (most recent call last): File "/usr/share/virt-who/virtwho.py", line 111, in _send virtualGuests = self._readGuests(config) File "/usr/share/virt-who/virtwho.py", line 147, in _readGuests return virt.getHostGuestMapping() File "/usr/share/virt-who/virt/esx/esx.py", line 301, in getHostGuestMapping self.scan() File "/usr/share/virt-who/virt/esx/esx.py", line 139, in scan raise virt.VirtError(str(e)) VirtError: <urlopen error timed out> 2014-10-31 15:53:09,253 [ERROR] @esx.py:138 - Unable to connect to ESX Traceback (most recent call last): File "/usr/share/virt-who/virt/esx/esx.py", line 136, in scan self.client = suds.client.Client("%s/sdk/vimService.wsdl" % self.url) File "/usr/lib/python2.7/site-packages/suds/client.py", line 112, in __init__ self.wsdl = reader.open(url) File "/usr/lib/python2.7/site-packages/suds/reader.py", line 152, in open d = self.fn(url, self.options) File "/usr/lib/python2.7/site-packages/suds/wsdl.py", line 136, in __init__ d = reader.open(url) File "/usr/lib/python2.7/site-packages/suds/reader.py", line 79, in open d = self.download(url) File "/usr/lib/python2.7/site-packages/suds/reader.py", line 95, in download fp = self.options.transport.open(Request(url)) File "/usr/lib/python2.7/site-packages/suds/transport/https.py", line 60, in open return HttpTransport.open(self, request) File "/usr/lib/python2.7/site-packages/suds/transport/http.py", line 62, in open return self.u2open(u2request) File "/usr/lib/python2.7/site-packages/suds/transport/http.py", line 118, in u2open return url.open(u2request, timeout=tm) File "/usr/lib64/python2.7/urllib2.py", line 404, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 422, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1224, in https_open return self.do_open(httplib.HTTPSConnection, req) File "/usr/lib64/python2.7/urllib2.py", line 1186, in do_open raise URLError(err) URLError: <urlopen error timed out> 2014-10-31 15:53:09,254 [ERROR] @virtwho.py:121 - Unable to recover, retry in 10 seconds. 2014-10-31 15:53:14,849 [INFO] @subscriptionmanager.py:116 - Sending update in hosts-to-guests mapping: {aee4ff00-8c33-11e2-994a-6c3be51d959a: [4229ae42-1501-d014-0b4f-e38e46960ce3, 564d30c2-464f-81ad-1db6-6ec620dd7465]} Expected results: No matter which vcenter can't reachable, it shouldn't effect virt-who send the host/guest associate of the reachable ones. Additional info: