Bug 1296936 - Vm.status() causes crash of MoM GuestManager
Vm.status() causes crash of MoM GuestManager
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm (Show other bugs)
3.5.6
Unspecified Unspecified
high Severity high
: ovirt-3.6.3
: 3.6.3
Assigned To: Francesco Romani
Shira Maximov
: ZStream
Depends On:
Blocks: 1298190
  Show dependency treegraph
 
Reported: 2016-01-08 08:40 EST by Roman Hodain
Modified: 2016-03-28 08:34 EDT (History)
12 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
With this update, the MOM component no longer fails to enforce QoS policies, KSM, and memory ballooning.
Story Points: ---
Clone Of:
: 1298190 (view as bug list)
Environment:
Last Closed: 2016-03-09 14:46:56 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 51655 master MERGED migration: use context manager for semaphore 2016-01-13 14:45 EST
oVirt gerrit 51656 master MERGED virt: safer handling of migration parameters 2016-01-13 14:49 EST
oVirt gerrit 51827 ovirt-3.6 MERGED migration: use context manager for semaphore 2016-01-18 03:31 EST
oVirt gerrit 51828 ovirt-3.6 MERGED virt: safer handling of migration parameters 2016-01-18 06:11 EST
oVirt gerrit 51829 ovirt-3.6 MERGED vm: safer handling of conf in restore 2016-01-18 06:11 EST
oVirt gerrit 52341 ovirt-3.5 MERGED migration: use context manager for semaphore 2016-01-19 04:16 EST
oVirt gerrit 52342 ovirt-3.5 MERGED virt: safer handling of migration parameters 2016-01-19 04:17 EST
oVirt gerrit 52343 ovirt-3.5 MERGED vm: safer handling of conf in restore 2016-01-19 04:18 EST

  None (edit)
Description Roman Hodain 2016-01-08 08:40:49 EST
Description of problem:
calling status on an instance of class Vm(object) throws the following exception:

2016-01-05 23:41:17,725 - mom.GuestManager - ERROR - Guest Manager crashed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/mom/GuestManager.py", line 114, in run
  File "/usr/lib/python2.6/site-packages/mom/HypervisorInterfaces/vdsmInterface.py", line 75, in getVmList
  File "/usr/share/vdsm/API.py", line 1380, in getVMList
  File "/usr/share/vdsm/API.py", line 1370, in reportedStatus
  File "/usr/share/vdsm/virt/vm.py", line 2817, in status
  File "/usr/share/vdsm/virt/vm.py", line 2817, in <genexpr>
RuntimeError: dictionary changed size during iteration
2016-01-05 23:41:19,843 - mom - ERROR - Thread 'GuestManager' has exited

Version-Release number of selected component (if applicable):
    vdsm-4.16.20-1.el6ev.x86_64

How reproducible:
    Randomly (race condition)

Steps to Reproduce:
Most probably by forcing massive live migration

Actual results:
the mentioned exception

Expected results:
Exception cannot raise

Additional info:

In this case the exception causes malfunctioning MoM as the GuestManager crashes due to this exception.
Comment 12 Francesco Romani 2016-01-13 04:19:54 EST
patch http://gerrit.ovirt.org/51657 is related but not required to fix this bug; removed.
Comment 16 Shira Maximov 2016-02-23 03:32:29 EST
verified on  Red Hat Enterprise Virtualization Manager Version: 3.6.3.2-0.1.el6  
mom-0.5.2-1.el7ev.noarch
vdsm-4.17.21-0.el7ev.noarch


verification steps: 
1. Create a pool of 20 VMs  (The hosts where very loaded, 5 Vms on each core) 
2. Migrate all the 20 VMs 
3. Check in mom logs

To be sure, I repeated this steps several times.
I didn't see this exception.
Comment 18 errata-xmlrpc 2016-03-09 14:46:56 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0362.html

Note You need to log in before you can comment on or make changes to this bug.