Created attachment 500614 [details]
vdsm log

Description of problem:
during _recoverExistingVms there is 5 seconds sleep that if during this sleep time vm is migrated/destroyed there is an unhanded exception which preventing all the other vms to connect.
while self._enabled and self.vmContainer and \
                  not self.irs.getConnectedStoragePoolsList()['poollist']:
                time.sleep(5)

            for vmId in self.vmContainer.keys():
                # Do not prepare volumes when system goes down
                if self._enabled:
                    self.vmContainer[vmId].preparePaths()

Thread-662::DEBUG::2011-05-24 16:15:18,221::clientIF::49::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Down', 'timeOffset': '0', 'vmId': '7a61d6c8-3df2-4551-b4c4-69260affc29f', 'exitMessage': 'Migration succeeded', 'exitCode': 0}]}
Thread-551::INFO::2011-05-24 16:15:18,224::libvirtvm::228::vm.Vm::(run) vmId=`18c4eaa5-ef78-44f8-b31b-f4a64a332d45`::Migration Progress: 10 seconds elapsed, 63% of data processed, 63% of mem processed
Thread-551::INFO::2011-05-24 16:15:18,236::libvirtvm::228::vm.Vm::(run) vmId=`18c4eaa5-ef78-44f8-b31b-f4a64a332d45`::Migration Progress: 10 seconds elapsed, 63% of data processed, 63% of mem processed
Thread-663::DEBUG::2011-05-24 16:15:18,241::clientIF::44::vds::(wrapper) [10.16.144.114]::call destroy with ('7a61d6c8-3df2-4551-b4c4-69260affc29f',) {}
Thread-663::INFO::2011-05-24 16:15:18,242::clientIF::443::vds::(destroy) vmContainerLock aquired by vm 7a61d6c8-3df2-4551-b4c4-69260affc29f
Thread-663::DEBUG::2011-05-24 16:15:18,243::libvirtvm::1160::vm.Vm::(destroy) vmId=`7a61d6c8-3df2-4551-b4c4-69260affc29f`::destroy Called
Thread-663::INFO::2011-05-24 16:15:18,244::libvirtvm::1123::vm.Vm::(releaseVm) vmId=`7a61d6c8-3df2-4551-b4c4-69260affc29f`::Release VM resources
Thread-663::WARNING::2011-05-24 16:15:18,244::vm::552::vm.Vm::(_set_lastStatus) vmId=`7a61d6c8-3df2-4551-b4c4-69260affc29f`::trying to set state to Powering down when already Down
Thread-663::DEBUG::2011-05-24 16:15:18,245::utils::471::vm.Vm::(stop) vmId=`7a61d6c8-3df2-4551-b4c4-69260affc29f`::Stop statistics collection

clientIFinit::ERROR::2011-05-24 16:20:06,951::clientIF::1183::vds::(_recoverExistingVms) Traceback (most recent call last):
  File "/usr/share/vdsm/clientIF.py", line 1181, in _recoverExistingVms
    self.vmContainer[vmId].preparePaths()
KeyError: '7a61d6c8-3df2-4551-b4c4-69260affc29f'



Version-Release number of selected component (if applicable):
vdsm-4.9-67.el6.x86_64

How reproducible:


Steps to Reproduce:
1.restart vdsm and migrate or destroy vm during the 5 seconds sleep (will be hard to reproduce)
2.
3.
  
Actual results:


Expected results:


Additional info: