Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
virt-who eating up the CPU and /var/log/message filled up with logs.
Virt-who is executing send_data and getHostGuestMapping function repeatedly and eating up CPU.
~~~
Apr 14 10:06:27 rhn01 /usr/bin/virt-who: 2020-04-14 10:06:27,770 [virtwho.main INFO] MainProcess(5418):Thread-2 @virt.py:_send_data:981 - Report for config "virt-who-config-3" gathered, placing in datastore
Apr 14 10:06:27 rhn01 /usr/bin/virt-who: 2020-04-14 10:06:27,949 [virtwho.main INFO] MainProcess(5418):Thread-2 @virt.py:_send_data:981 - Report for config "virt-who-config-3" gathered, placing in datastore
Apr 14 10:06:28 rhn01 /usr/bin/virt-who: 2020-04-14 10:06:28,457 [virtwho.main INFO] MainProcess(5418):Thread-2 @virt.py:_send_data:981 - Report for config "virt-who-config-3" gathered, placing in datastore
Apr 14 10:06:30 rhn01 /usr/bin/virt-who: 2020-04-14 10:06:30,805 [virtwho.main INFO] MainProcess(5418):Thread-2 @virt.py:_send_data:981 - Report for config "virt-who-config-3" gathered, placing in datastore
Apr 14 10:06:34 rhn01 /usr/bin/virt-who: 2020-04-14 10:06:34,375 [virtwho.main INFO] MainProcess(5418):Thread-2 @virt.py:_send_data:981 - Report for config "virt-who-config-3" gathered, placing in datastore
~~~
- It looks like config "virt-who-config-3" gets stuck in an infinite loop. Above logs are continuously generated until we stop the virt-who service.
- Tried to reinstall and downgrade the virt-who package but no success. Also, confirmed that none of the config files is modified # rpm -V virt-who
- This issue is only with the virt-who-config-3, If we remove the virt-who-config-3 then virt-who services work properly with virt-who-config-2(which contains different vcenter).
- virt-who -o (oneshot) working fine with "virt-who-config-3"
- There is nothing special or any extra parameter in virt-who-config-3.conf which can cause this.
Not sure but I think virt-who get stuck in this while loop - https://github.com/candlepin/virt-who/blob/e56d841f742f46eed853c8c106c7ed85eeb4b3e4/virtwho/virt/esx/esx.py#L153 and because of this self._send_data the method is executed and above logs are printed https://github.com/candlepin/virt-who/blob/e56d841f742f46eed853c8c106c7ed85eeb4b3e4/virtwho/virt/virt.py#L977
Version-Release number of selected component (if applicable):
- virt-who-0.24.7-1.el7.noarch
- Red Hat Enterprise Linux Server release 7.7 (Maipo)
Actual results:
"virt-who-config-3" gets stuck in an infinite loop. This works with oneshot, the problem is with a daemon.
Expected results:
virt-who service should work properly without repeatedly executing send_data and getHostGuestMapping function https://github.com/candlepin/virt-who/blob/6e8616350212b2d694e2e6a637ca4f7ff4cac2ef/virtwho/virt/esx/esx.py#L206
- Can it be the issue related to specific Vcenter?
Hello yuefliu ,
Thanks for the update.
Please configure virt-who using virt-who configuration from satellite itself to reproduce the issue on test ENV.
https://bootp-73-3-198.eng.pek2.redhat.com/users/login >> Infrastructure >> Virt-who Configurations >> create a new config and then proceed with testing.
Comment 15Sudhir Mallamprabhakara
2020-04-29 12:32:08 UTC
@Rehana - Reading thru the comments, this sounds like a load/performance issue. Can we have someone from the perf team try to reproduce this?
-Sudhir
(In reply to Sudhir Mallamprabhakara from comment #15)
> @Rehana - Reading thru the comments, this sounds like a load/performance
> issue. Can we have someone from the perf team try to reproduce this?
>
> -Sudhir
Hi Sudhir,
Thanks for reaching out , our dev team will be looking in this issue in the current sprint. Couple of our engineers are out on training this week , so I see this card mentioned in comment 14 picking up later this week. It would really helpful if you can help us with a test team who could helps us with the investigation .
thanks,
Rehana
Hello William,
Thanks for the update.
Please help me with virt-who build 0.28.5-1 package to test it and post you with result.
Awaiting for the attachments.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (virt-who bug fix update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2020:3865