Bug 1295644
| Summary: | virt-who can't send two hypervisors' host/guests info at the same time, need to wait for 60s | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Eko <hsun> |
| Component: | virt-who | Assignee: | Radek Novacek <rnovacek> |
| Status: | CLOSED ERRATA | QA Contact: | Eko <hsun> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.8 | CC: | csnyder, ovasik, rbalakri, sgao, shihliu |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | virt-who-0.16-6.el6 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-05-10 23:57:12 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This is now expected behaviour. In order to prevent flooding the candlepin server, virt-who now limits how often it sends reports. By default reports are not sent more often than once per minute. For details please see: https://github.com/virt-who/virt-who/wiki/Interval-changes Hi radek, I agree with you, if the virt-who is running now, it's ok to check the reports update per minute. but when restart virt-who service, virt-who should send all the hypervisors json at the same time. Configure two hypervisors as example, such as: hypervisor_1 (kvm): host/guests json hypervisor_2 (esx): host/guests json after restart virt-who service, hypervisor_1 (kvm) will be sent immediately, but hypervisor_2 (esx) will be delaied 60 seconds. Sending all the hypervisor reports at the same time creates a lot of load in the candlepin and that is something we generally don't want to do. Although I agree that from user perspective it would be better to send both reports asap, that's something that candlepin doesn't want. if the customer create 100 config files in /etc/virt-who.d/, it will take 100 minutes to open all the config files and send the host/guests json. if change the Interval value to 120 seconds, it will take 200 minutes to polling one time. Chris, what do you think would be correct behavior in this case? Let the customer wait for initial run too long doesn't seem right. OTOH the load for candlepin can be quite high, given how big environments customers have. I don't know how many configs our users have. I don't think I saw more than 2 configs on one virt-who instance, but I'm not in touch with our customers. Eko, do you have an idea? Chris, how about sending the first batch immediately when virt-who is starting and apply the rate limit after one report for each config is sent. It would be better from user perspective and candlepin can still reply with 429 - then virt-who will delay all future reports (even if the first batch isn't finished). Is this solution acceptable? Radek, That sounds reasonable to me. Thanks, Chris Fixed in virt-who-0.16-6.el6. Verified it on virt-who-0.16-7.el6.noarch since virt-who can get multi esx/rhevm/hyperv/remote libvirt mapping info and send it to server side immediately. Therefore, verify it. Verified version: virt-who-0.16-7.el6.noarch python-rhsm-1.16.6-1.el6.x86_64 subscription-manager-1.16.8-4.el6.x86_64 Verified process: 1. Register system to satellite6.1 2. Configure two type of mode in /etc/virt-who.d/ [root@intel-piketon-01 ~]# cat /etc/virt-who.d/virt [test-rhevm1] type=rhevm server=https://10.73.2.65:443 username=admin@internal password=redhat owner=ACME_Corporation env=Library [root@intel-piketon-01 ~]# cat /etc/virt-who.d/esx [test-esx1] type=esx server=10.73.2.95 username=Administrator password=Welcome1! owner=ACME_Corporation env=Library 3. Restart virt-who service and check virt-who's log [root@intel-piketon-01 ~]# service virt-who restart && tail -f /var/log/rhsm/rhsm.log Stopping virt-who: [ OK ] Starting virt-who: [ OK ] .............. 2016-03-16 02:21:04,047 [virtwho.init INFO] MainProcess(27419):MainThread @virtwho.py:parseOptions:637 - Using reporter_id='intel-piketon-01.lab.bos.redhat.com' 2016-03-16 02:21:04,051 [virtwho.init DEBUG] MainProcess(27419):MainThread @virtwho.py:__init__:125 - Using config named 'test-esx1' 2016-03-16 02:21:04,051 [virtwho.init DEBUG] MainProcess(27419):MainThread @virtwho.py:__init__:125 - Using config named 'test-rhevm1' 2016-03-16 02:21:04,051 [virtwho.init INFO] MainProcess(27419):MainThread @virtwho.py:main:729 - Using configuration "test-esx1" ("esx" mode) 2016-03-16 02:21:04,051 [virtwho.init INFO] MainProcess(27419):MainThread @virtwho.py:main:729 - Using configuration "test-rhevm1" ("rhevm" mode) 2016-03-16 02:21:04,067 [virtwho.main DEBUG] MainProcess(27421):MainThread @virtwho.py:run:231 - Starting infinite loop with 60 seconds interval 2016-03-16 02:21:04,186 [virtwho.test-esx1 DEBUG] Esx-1(27430):MainThread @virt.py:run:358 - Virt backend 'test-esx1' started 2016-03-16 02:21:04,187 [virtwho.test-esx1 DEBUG] Esx-1(27430):MainThread @esx.py:_prepare:55 - Log into ESX 2016-03-16 02:21:04,190 [virtwho.test-rhevm1 DEBUG] RhevM-2(27432):MainThread @virt.py:run:358 - Virt backend 'test-rhevm1' started 2016-03-16 02:21:07,735 [virtwho.test-esx1 DEBUG] Esx-1(27430):MainThread @esx.py:_prepare:58 - Creating ESX event filter 2016-03-16 02:21:08,435 [virtwho.test-rhevm1 DEBUG] RhevM-2(27432):MainThread @virt.py:enqueue:351 - Report for config "test-rhevm1" gathered, putting to queue for sending 2016-03-16 02:21:08,453 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:_connect:121 - Authenticating with certificate: /etc/pki/consumer/cert.pem 2016-03-16 02:21:08,642 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:162 - Checking if server has capability 'hypervisor_async' 2016-03-16 02:21:08,785 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:174 - Server does not have 'hypervisors_async' capability 2016-03-16 02:21:08,786 [virtwho.main INFO] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:185 - Sending update in hosts-to-guests mapping for config "test-rhevm1": 2 hypervisors and 1 guests found 2016-03-16 02:21:08,786 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:186 - Host-to-guest mapping: { "fd26642f-62f5-4075-adce-3359e5a8bd10": [ { "guestId": "a11f0b29-b430-4a53-978a-cae879ad1ef3", "state": 1, "attributes": { "active": 1, "hypervisorVersion": "", "virtWhoType": "rhevm", "hypervisorType": "qemu" } } ], "d64f48cc-5fda-45c3-a307-fa42ef1864c7": [] } 2016-03-16 02:21:09,057 [virtwho.main DEBUG] MainProcess(27421):MainThread @virtwho.py:send_report:161 - Report for config "test-rhevm1" sent 2016-03-16 02:21:11,114 [virtwho.test-esx1 DEBUG] Esx-1(27430):MainThread @virt.py:enqueue:351 - Report for config "test-esx1" gathered, putting to queue for sending 2016-03-16 02:21:11,118 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:_connect:121 - Authenticating with certificate: /etc/pki/consumer/cert.pem 2016-03-16 02:21:11,286 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:162 - Checking if server has capability 'hypervisor_async' 2016-03-16 02:21:11,616 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:174 - Server does not have 'hypervisors_async' capability 2016-03-16 02:21:11,616 [virtwho.main INFO] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:185 - Sending update in hosts-to-guests mapping for config "test-esx1": 1 hypervisors and 1 guests found 2016-03-16 02:21:11,617 [virtwho.main DEBUG] MainProcess(27421):MainThread @subscriptionmanager.py:hypervisorCheckIn:186 - Host-to-guest mapping: { "86b2bd00-8bad-11e2-87f4-6c3be514699d": [ { "guestId": "42060bb0-044b-dd8c-c111-361bb5b2f78c", "state": 5, "attributes": { "active": 0, "hypervisorVersion": "6.0.0", "virtWhoType": "esx", "hypervisorType": "VMware ESXi" } } ] } 2016-03-16 02:21:11,917 [virtwho.main DEBUG] MainProcess(27421):MainThread @virtwho.py:send_report:161 - Report for config "test-esx1" sent Result: Virt-who can send multi host/guest mapping info to server immediately. Note: Configure multi server/mode in /etc/sysconfig/virt-who and /etc/virt-who.d/, virt-who also can send mapping info simultaneously. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0859.html |
Description of problem: If I configured two hypervisors modes, such as libvirt and esx, after virt-who restart, the mapping info can't be sent at the same time. It will send libvirt host/guests info firstly after 60s, will send the esx host/guests info Version-Release number of selected component (if applicable): virt-who-0.16-1.el6.noarch How reproducible: always Steps to Reproduce: 1. prepare two different hypervisors mode in /etc/virt-who.d, such as: # vi /etc/virt-who.d/libvirt.conf [test-libvirt] type=libvirt server=10.66.144.8 username=root password=redhat owner=ACME_Corporation env=Library # vi /etc/virt-who.d/esx.conf [test-esx] type=esx server=10.73.2.15 username=Administrator password=Welcome1! owner=ACME_Corporation env=Library 2. restart virt-who service # /etc/init.d/virt-who restart 3. check rhsm.log 2016-01-05 13:31:48,335 [virtwho.init DEBUG] MainProcess(21368):MainThread @virtwho.py:__init__:144 - Using config named 'test-esx' 2016-01-05 13:31:48,335 [virtwho.init DEBUG] MainProcess(21368):MainThread @virtwho.py:__init__:144 - Using config named 'test-libvirt' 2016-01-05 13:31:48,335 [virtwho.init INFO] MainProcess(21368):MainThread @virtwho.py:main:811 - Using configuration "test-esx" ("esx" mode) 2016-01-05 13:31:48,335 [virtwho.init INFO] MainProcess(21368):MainThread @virtwho.py:main:811 - Using configuration "test-libvirt" ("libvirt" mode) 2016-01-05 13:31:48,348 [virtwho.main DEBUG] MainProcess(21370):MainThread @virtwho.py:run:287 - Starting infinite loop with 60 seconds interval 2016-01-05 13:31:48,444 [virtwho.test-esx DEBUG] Esx-1(21378):MainThread @virt.py:run:338 - Virt backend 'test-esx' started 2016-01-05 13:31:48,444 [virtwho.test-esx DEBUG] Esx-1(21378):MainThread @esx.py:_prepare:55 - Log into ESX 2016-01-05 13:31:48,446 [virtwho.test-libvirt DEBUG] Libvirtd-2(21380):MainThread @virt.py:run:338 - Virt backend 'test-libvirt' started 2016-01-05 13:31:48,447 [virtwho.test-libvirt INFO] Libvirtd-2(21380):MainThread @libvirtd.py:_get_url:124 - Protocol is not specified in libvirt url, using qemu+ssh:// 2016-01-05 13:31:48,447 [virtwho.test-libvirt INFO] Libvirtd-2(21380):MainThread @libvirtd.py:_get_url:135 - Libvirt path is not specified in the url, using /system 2016-01-05 13:31:48,447 [virtwho.test-libvirt INFO] Libvirtd-2(21380):MainThread @libvirtd.py:_connect:156 - Using libvirt url: qemu+ssh://root.144.8/system?no_tty=1 2016-01-05 13:31:48,790 [virtwho.test-libvirt DEBUG] Libvirtd-2(21380):MainThread @libvirtd.py:_listDomains:228 - Virtual machine found: rhel7.2-satellite: 8d3f130d-b8f5-4b5e-3a3b-5d07483a615c 2016-01-05 13:31:48,796 [virtwho.test-libvirt DEBUG] Libvirtd-2(21380):MainThread @libvirtd.py:_listDomains:234 - Virtual machine found: rhel7.2: cb33ddce-fd1e-0e6a-7e7f-d3c6fca67ede 2016-01-05 13:31:48,796 [virtwho.test-libvirt DEBUG] Libvirtd-2(21380):MainThread @libvirtd.py:_listDomains:238 - Libvirt domains found: 8d3f130d-b8f5-4b5e-3a3b-5d07483a615c, cb33ddce-fd1e-0e6a-7e7f-d3c6fca67ede 2016-01-05 13:31:48,796 [virtwho.test-libvirt DEBUG] Libvirtd-2(21380):MainThread @virt.py:enqueue:331 - Report gathered, putting to queue for sending 2016-01-05 13:31:48,799 [virtwho.main DEBUG] MainProcess(21370):MainThread @virtwho.py:update_report_to_send:274 - Report for config "test-libvirt" updated 2016-01-05 13:31:48,801 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:_connect:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem 2016-01-05 13:31:48,794 [virtwho.test-esx DEBUG] Esx-1(21378):MainThread @esx.py:_prepare:58 - Creating ESX event filter 2016-01-05 13:31:49,001 [virtwho.test-esx DEBUG] Esx-1(21378):MainThread @virt.py:enqueue:331 - Report gathered, putting to queue for sending 2016-01-05 13:31:49,002 [virtwho.test-esx DEBUG] Esx-1(21378):MainThread @esx.py:_run:150 - Waiting for ESX changes 2016-01-05 13:31:49,031 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:149 - Checking if server has capability 'hypervisor_async' 2016-01-05 13:31:49,513 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:161 - Server does not have 'hypervisors_async' capability 2016-01-05 13:31:49,513 [virtwho.main INFO] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:170 - Sending update in hosts-to-guests mapping: 1 hypervisors and 2 guests found 2016-01-05 13:31:49,514 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:171 - Host-to-guest mapping: { "80804c56-82fb-e111-a260-b4b52fcb471e": [ { "guestId": "8d3f130d-b8f5-4b5e-3a3b-5d07483a615c", "state": 1, "attributes": { "active": 1, "hypervisorVersion": "0.12.1", "virtWhoType": "libvirt", "hypervisorType": "QEMU" } }, { "guestId": "cb33ddce-fd1e-0e6a-7e7f-d3c6fca67ede", "state": 5, "attributes": { "active": 0, "hypervisorVersion": "0.12.1", "virtWhoType": "libvirt", "hypervisorType": "QEMU" } } ] } 2016-01-05 13:31:49,827 [virtwho.main DEBUG] MainProcess(21370):MainThread @virtwho.py:send_current_report:206 - Report for config "test-libvirt" sent 2016-01-05 13:31:49,829 [virtwho.main DEBUG] MainProcess(21370):MainThread @virtwho.py:update_report_to_send:274 - Report for config "test-esx" updated ========== only send libvirt host/guests info as above ========== ========== after 60s, will send esx host/guest info as below ========== 2016-01-05 13:32:48,853 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:_connect:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem 2016-01-05 13:32:49,067 [virtwho.test-esx DEBUG] Esx-1(21378):MainThread @esx.py:_run:150 - Waiting for ESX changes 2016-01-05 13:32:49,089 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:149 - Checking if server has capability 'hypervisor_async' 2016-01-05 13:32:49,317 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:161 - Server does not have 'hypervisors_async' capability 2016-01-05 13:32:49,317 [virtwho.main INFO] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:170 - Sending update in hosts-to-guests mapping: 2 hypervisors and 3 guests found 2016-01-05 13:32:49,318 [virtwho.main DEBUG] MainProcess(21370):MainThread @subscriptionmanager.py:hypervisorCheckIn:171 - Host-to-guest mapping: { "4c4c4544-0048-5a10-8043-b5c04f393358": [ { "guestId": "42293e7d-ba7e-1cea-d897-d8a373b26925", "state": 5, "attributes": { "active": 0, "hypervisorVersion": "6.0.0", "virtWhoType": "esx", "hypervisorType": "VMware ESXi" } } ], "86b2bd00-8bad-11e2-87f4-6c3be514699d": [ { "guestId": "42098426-0d49-7dc6-0cb7-2331f149b8aa", "state": 3, "attributes": { "active": 1, "hypervisorVersion": "6.0.0", "virtWhoType": "esx", "hypervisorType": "VMware ESXi" } }, { "guestId": "4239b745-4369-40fd-569c-ba0d396654e9", "state": 5, "attributes": { "active": 0, "hypervisorVersion": "6.0.0", "virtWhoType": "esx", "hypervisorType": "VMware ESXi" } } ] } 2016-01-05 13:32:49,616 [virtwho.main DEBUG] MainProcess(21370):MainThread @virtwho.py:send_current_report:206 - Report for config "test-esx" sent Actual results: virt-who can't send two hypervisors' host/guests info at the same time, need to wait for 60s Expected results: after restart, virt-who should send all the hypervisors' host/guests info at the same time Additional info: