Bug 1298527

Description of problem:
Noticed while verifying https://bugzilla.redhat.com/show_bug.cgi?id=1296936
Noticed under sustained migration load, but can happen in other cases

If MOM queries Vdsm when a VM just migrated, or in general just died but it is not yet collected by Engine, we could find

2016-01-14 09:40:19,043 - mom.Monitor - ERROR - Unexpected collection error
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mom/Monitor.py", line 95, in collect
    collected = c.collect()
  File "/usr/lib/python2.7/site-packages/mom/Collectors/GuestMemory.py", line 50, in collect
    stat = self.hypervisor_iface.getVmMemoryStats(self.uuid)
  File "/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmxmlrpcInterface.py", line 121, in getVmMemoryStats
    usage = int(vm['memUsage'])
KeyError: 'memUsage'

in MOM logs.

This should be little harm because the VM is already down, so MOM has not much to do here.

It is not Vdsm fault in this case because if Vm is down, a getVmStats call
must return ExitedVmStats (not RunningVmStats, see schema), and memUsage is not part of ExitedVmStats.

Version-Release number of selected component (if applicable):
# rpm -qi mom
Name        : mom
Version     : 0.5.1
Release     : 2.1.el7
# rpm -qi vdsm
Name        : vdsm
Version     : 4.17.17
Release     : 3.git4517968.el7.centos
(prerelease)

How reproducible:
Time-based

Steps to Reproduce:
1. start a bunch of VMs
2. migrate or stop/wait/restart VMs, to increse the chance of query while VM is down and not collected (VM.destroy() from Engine)
3. check MOM logs

Actual results:
KeyError in the log, error not gracefully handled

Expected results:
MOM gracefully handles this case

Additional info:
To fix, it should be enough to enhance 
mom/HypervisorInterfaces/vdsmxmlrpcInterface.py:getVmStats like

diff --git a/mom/HypervisorInterfaces/vdsmxmlrpcInterface.py b/mom/HypervisorInterfaces/vdsmxmlrpcInterface.py
index 63b6d0d..950a6b6 100644
--- a/mom/HypervisorInterfaces/vdsmxmlrpcInterface.py
+++ b/mom/HypervisorInterfaces/vdsmxmlrpcInterface.py
@@ -93,15 +93,23 @@ class XmlRpcVdsmInterface(HypervisorInterface):
             vms[vm['vmId']] = vm
         return vms
 
-    def getVmStats(self, vmId):
-        return self.getAllVmStats()[vmId]
-
     def _vmIsRunning(self, vm):
         if vm['status'] == 'Up':
             return True
         else:
             return False
 
+    def getVmStats(self, vmId):
+        try:
+            vm = self.getAllVmStats()[vmId]
+        except KeyError as e:
+            raise HypervisorInterfaceError("VM %s does not exist" % uuid)
+        else:
+            if self._vmIsRunning(vm):
+                return vm
+            raise HypervisorInterfaceError(
+                "VM %s not in Up state" % uuid)