Bug 1263950

Summary: Migrating VM get stuck when killed on destination host
Product: Red Hat Enterprise Virtualization Manager Reporter: Arik <ahadas>
Component: vdsmAssignee: Dan Kenigsberg <danken>
Status: CLOSED NOTABUG QA Contact: Ilanit Stein <istein>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: bazulay, ecohen, gklein, lsurette, ycui, yeylon
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-17 08:17:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
VDSM log on source
none
VDSM log on destination none

Description Arik 2015-09-17 07:35:47 UTC 
Description of problem:
Killing the qemu process on the destination host while the VM is migrating causes the VM to get stuck in Migration Source* in VDSM although in the source host the VM is 'running' in libvirt.

Version-Release number of selected component (if applicable):


How reproducible:
VDSM: 4.17.5
Libvirt: 1.2.18 (1.fc22)
QEMU: 2.4.0 (2.fc22)

Steps to Reproduce:
1. Migrate a VM
2. Kill the qemu process on the destination host
3.

Actual results:
VM stays in MigratingFrom status in the engine.

vdsClient 0 list table:
9397bed3-70a4-48bc-a0e5-77ff5a212557   1812  mmm  Migration Source* 

virsh > list
 Id    Name                           State
----------------------------------------------------
 2     mmm                            running

Expected results:
The VM should return to status UP on the source.

Additional info:
The following exception is repeating in VDSM log:
virt.vm::(_getRunningVmStats) vmId=`9397bed3-70a4-48bc-a0e5-77ff5a212557`::Error fetching vm stats
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 1384, in _getRunningVmStats
    vm_sample.interval)
  File "/usr/share/vdsm/virt/vmstats.py", line 43, in produce
    disks(vm, stats, first_sample, last_sample, interval)
  File "/usr/share/vdsm/virt/vmstats.py", line 262, in disks
    interval))
  File "/usr/share/vdsm/virt/vmstats.py", line 291, in _disk_rate
    (last_sample['block.%d.rd.bytes' % last_index] -
KeyError: 'block.1.rd.bytes'
Comment 1 Arik 2015-09-17 07:40:23 UTC 
Created attachment 1074316 [details]
VDSM log on source
Comment 2 Arik 2015-09-17 07:41:31 UTC 
Created attachment 1074317 [details]
VDSM log on destination
Comment 3 Arik 2015-09-17 08:17:48 UTC 
Apparently a problem in libvirt.