Bug 672208

Summary: [vdsm] When migrating multiple VMs some getting stuck in Paused status.
Product: Red Hat Enterprise Linux 6 Reporter: David Naori <dnaori>
Component: vdsmAssignee: Eduardo Warszawski <ewarszaw>
Status: CLOSED ERRATA QA Contact: yeylon <yeylon>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.1CC: abaron, bazulay, danken, dnaori, hateya, iheim, mgoldboi, srevivo
Target Milestone: rc   
Target Release: 6.1   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 07:04:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Naori 2011-01-24 11:52:38 UTC
Description of problem:

When migrating multiple VMs some getting stuck in Paused status on the source host. 

vdsm log:

Thread-612::ERROR::2011-01-24 13:05:41,728::vm::453::vds.vmlog.67cd8b9a-f684-46d2-b2a0-c7fa62b4fe3f::(run) Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 446, in run
    self._finishSuccessfully()
  File "/usr/share/vdsm/vm.py", line 373, in _finishSuccessfully
    self._vm.setDownStatus(NORMAL, "Migration succeeded")
  File "/usr/share/vdsm/vm.py", line 1743, in setDownStatus
    self.lastStatus = 'Down'
  File "/usr/share/vdsm/vm.py", line 576, in _set_lastStatus
    self.saveState()
  File "/usr/share/vdsm/libvirtvm.py", line 918, in saveState
    vm.Vm.saveState(self)
  File "/usr/share/vdsm/vm.py", line 1227, in saveState
    pickle.dump(toSave, f)
  File "/usr/lib64/python2.6/pickle.py", line 1362, in dump
    Pickler(file, protocol).dump(obj)
  File "/usr/lib64/python2.6/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.6/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib64/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.6/pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "/usr/lib64/python2.6/pickle.py", line 615, in _batch_appends
    save(x)
  File "/usr/lib64/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.6/pickle.py", line 661, in _batch_setitems
    for k, v in items:
RuntimeError: dictionary changed size during iteration

Versions:
-rhel 6.1 based hosts
-vdsm-4.9-43.1
-libvirt-0.8.7-3
-rhevm 2.3

repo steps:

1) Create 30 VMs from pool.
2) Migrate all of them together from one host to another.

Comment 2 Dan Kenigsberg 2011-02-10 08:23:49 UTC
this might be related to bug 660598

Comment 4 David Naori 2011-02-28 16:52:54 UTC
Verified on vdsm-4.9-51.el6.x86_64

Comment 5 errata-xmlrpc 2011-12-06 07:04:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html