Bug 1296936

Summary: Vm.status() causes crash of MoM GuestManager
Product: Red Hat Enterprise Virtualization Manager Reporter: Roman Hodain <rhodain>
Component: vdsmAssignee: Francesco Romani <fromani>
Status: CLOSED ERRATA QA Contact: Shira Maximov <mshira>
Severity: high Docs Contact:
Priority: high    
Version: 3.5.6CC: bazulay, fromani, gklein, inetkach, lsurette, melewis, michal.skrivanek, mkalinin, sbonazzo, ycui, yeylon, ykaul
Target Milestone: ovirt-3.6.3Keywords: ZStream
Target Release: 3.6.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
With this update, the MOM component no longer fails to enforce QoS policies, KSM, and memory ballooning.
Story Points: ---
Clone Of:
: 1298190 (view as bug list) Environment:
Last Closed: 2016-03-09 19:46:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1298190    

Description Roman Hodain 2016-01-08 13:40:49 UTC
Description of problem:
calling status on an instance of class Vm(object) throws the following exception:

2016-01-05 23:41:17,725 - mom.GuestManager - ERROR - Guest Manager crashed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/mom/GuestManager.py", line 114, in run
  File "/usr/lib/python2.6/site-packages/mom/HypervisorInterfaces/vdsmInterface.py", line 75, in getVmList
  File "/usr/share/vdsm/API.py", line 1380, in getVMList
  File "/usr/share/vdsm/API.py", line 1370, in reportedStatus
  File "/usr/share/vdsm/virt/vm.py", line 2817, in status
  File "/usr/share/vdsm/virt/vm.py", line 2817, in <genexpr>
RuntimeError: dictionary changed size during iteration
2016-01-05 23:41:19,843 - mom - ERROR - Thread 'GuestManager' has exited

Version-Release number of selected component (if applicable):
    vdsm-4.16.20-1.el6ev.x86_64

How reproducible:
    Randomly (race condition)

Steps to Reproduce:
Most probably by forcing massive live migration

Actual results:
the mentioned exception

Expected results:
Exception cannot raise

Additional info:

In this case the exception causes malfunctioning MoM as the GuestManager crashes due to this exception.

Comment 12 Francesco Romani 2016-01-13 09:19:54 UTC
patch http://gerrit.ovirt.org/51657 is related but not required to fix this bug; removed.

Comment 16 Shira Maximov 2016-02-23 08:32:29 UTC
verified on  Red Hat Enterprise Virtualization Manager Version: 3.6.3.2-0.1.el6  
mom-0.5.2-1.el7ev.noarch
vdsm-4.17.21-0.el7ev.noarch


verification steps: 
1. Create a pool of 20 VMs  (The hosts where very loaded, 5 Vms on each core) 
2. Migrate all the 20 VMs 
3. Check in mom logs

To be sure, I repeated this steps several times.
I didn't see this exception.

Comment 18 errata-xmlrpc 2016-03-09 19:46:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0362.html