Bug 1155679

Created redmine issue http://projects.theforeman.org/issues/8279 from this bug

This bug is addressed by upstream and will be fixed by rebase in RHEL-6.7.

Fixed by rebase to virt-who-0.12-1.el6.

Hi,
  we want to verify this bug but we don't have the environment to do the performance testing, we only have two ESX machines. we did a lot of research about how to do performance testing with two machines but we failed, so is there someone else can help us to do the performance testing?

Performance testing is typically done by Performance Engineering team https://home.corp.redhat.com/wiki/performance-engineering-rhel . I am not sure whether they can do the sort of testing you need, however we can at least ask Thomas.

@Thomas: may I ask you please to help xingge with the testing, if possible ?

Jan
     We do not have the infrastructure to test an environment like this. I do not know who has esx machines capable of doing this testing. 

Tom

Created attachment 1010250 [details]
virt-who.0-12-esx-rhsm.log

Tested in customer environment. Getting a large amount of noise in the logs now.

Setting VIRTWHO_DEBUG=0 in /etc/sysconfig/virt-who should reduce the noise. If not, I can remove some of the logging messages.

I've investigated the error messages more deeply and it looks like it's caused by the fact that ESXi/vCenter splits the guest list to more parts when it's big enough and virt-who only reads the first part and therefore it produces incomplete host/guest association. It probably fixes itself during next iteration, but it should still be corrected.

I'll make updated version for testing. Moving the bug to ASSIGNED for now.

Fixed in virt-who-0.12-4.el6.

Please try following build, it should fix the "noise" in the logs:

https://brewweb.devel.redhat.com/taskinfo?taskID=8949195

Hi William,

Would you mind to help us to verify it again on customer environment since we still don't have the environment to do the performance testing? Thanks.

Liushihui

As we do not have the environment to test this bug so I'll close this bug. If the bug shows again in the customer's environment please reopen this bug .

Thom Carlin reached out to the RHC project managers so they can contact the 2 customers to verify the bug, so reopen it and wait for the results

Hi Thom,
  Since this bug is listed in the virt-who errata [1] which is needed to push out asap along with the rhel6.7 release, and for now, in the errata [1] this bug is the only bug which is not verified, so could you please help coordinate to get this bug verified at your earliest convenience so that we can push out the errata timely? Thanks!

Summary:
As we haven't the env with many hosts/guests, we only check it with three hosts and one guests.we just verify the response time which virt-who get data from vcenter. the response time in the new version virt-who-0.12-10.el6.noarch is much less than the old version virt-who-0.10-8.el6.noarch.
About the problem of redundant noise log still need Thom Carlin's help to verify it. 

In the older version of virt-who-0.10-8.el6.noarch
The time of virt-who get data from vcenter is about 8s, please see the log as the following:
2015-07-02 11:15:22,370 [INFO]  @virtwho.py:442 - Using virt-who configuration: virt-who
2015-07-02 11:15:22,370 [DEBUG]  @virtwho.py:170 - Starting infinite loop with 3600 seconds interval
2015-07-02 11:15:22,519 [INFO]  @esx.py:134 - start scan
2015-07-02 11:15:27,898 [INFO]  @esx.py:164 - start get comput resource
2015-07-02 11:15:28,149 [INFO]  @esx.py:196 - start get host
2015-07-02 11:15:28,378 [INFO]  @esx.py:218 - start get vm
2015-07-02 11:15:30,875 [INFO]  @esx.py:325 - end scan
2015-07-02 11:15:30,875 [INFO]  @esx.py:327 - ==== time 8 
========================the response time is 8s============================
2015-07-02 11:15:30,913 [INFO]  @subscriptionmanager.py:119 - Sending update in hosts-to-guests mapping: {564d4b40-300e-98fd-fc79-5ae3c1547e2d: [], aee4ff00-8c33-11e2-994a-6c3be51d959a: [564dab7d-3b72-51a1-eeda-586036106892, 4227d611-0abc-fc5e-7538-07ebd83fa9ba, 421aa84b-a49e-e01c-fa72-0b570372dd9d, 564daa52-c518-b2f0-3c05-a343285910e1]}

In the new version of virt-who-0.12-10.el6.noarch:
The time of virt-who get data from vcenter is less than 1s, please see the log as the following:
2015-07-02 12:17:41,300 [INFO]  @virtwho.py:563 - Using configuration "env/cmdline" ("esx" mode)
2015-07-02 12:17:41,324 [DEBUG]  @virtwho.py:151 - Starting infinite loop with 3600 seconds interval
2015-07-02 12:17:41,463 [DEBUG]  @esx.py:56 - Log into ESX
2015-07-02 12:17:42,554 [DEBUG]  @esx.py:59 - Creating ESX event filter
2015-07-02 12:17:42,903 [INFO]  @esx.py:186 - end scan
2015-07-02 12:17:42,912 [INFO]  @esx.py:138 - ==== time 0 
========================the response time is 0s============================
2015-07-02 12:17:42,912 [DEBUG]  @esx.py:142 - Waiting for ESX changes
2015-07-02 12:17:42,923 [INFO]  @subscriptionmanager.py:124 - Sending upda
te in hosts-to-guests mapping: {564d4b40-300e-98fd-fc79-5ae3c1547e2d: [], aee4ff00-8c33-11e2-994a-6c3be51d959a: [564dab7d-3b72-51a1-eeda-586036106892, 4227d611-0abc-fc5e-7538-07ebd83fa9ba, 421aa84b-a49e-e01c-fa72-0b570372dd9d, 564daa52-c518-b2f0-3c05-a343285910e1], 564d9e7a-4128-92b6-7284-6335f6b399be: []}

Because we need to push Errata according to the schedule, change this bug to CLOSED status, and we will test the performance issue on RHEL7.2.

We don't have the environment to test the performance issue, but we do some analysis according to the source code, there are some enhancement for this issue according to the source code. closed this bug and continue to verify this issue on rhel7.2.