Hide Forgot
Description of problem: error looks as follows: Traceback (most recent call last): File "/usr/share/vdsm/utils.py", line 389, in collect statsFunction() File "/usr/share/vdsm/utils.py", line 266, in __call__ retValue = self._function(*args, **kwargs) File "/usr/share/vdsm/libvirtvm.py", line 157, in _sampleDiskLatency stats = _blockstatsParses(out) File "/usr/share/vdsm/libvirtvm.py", line 144, in _blockstatsParses 'flush_op':devStats['flush_operations'], KeyError: 'flush_operations' problem is in the following function: ------------------------------------------------------------------------------ def _sampleDiskLatency(self): if not self._vm._volumesPrepared: # Avoid queries from storage during recovery process return def _blockstatsParses(devList): # The json output looks like: # {u'return': [{u'device': u'drive-ide0-0-0', # u'stats': {u'rd_operations': 0, u'flush_total_time_ns': 0, u'wr_highest_offset': 0, u'rd_total_time_ns': 0, # u'rd_bytes': 0, u'wr_total_time_ns': 0, u'flush_operations': 0, u'wr_operations': 0, u'wr_bytes':0}, # u'parent': {u'stats': {u'rd_operations': 0, u'flush_total_time_ns': 0, u'wr_highest_offset': 0, # u'rd_total_time_ns': 0, u'rd_bytes': 0, u'wr_total_time_ns': 0, u'flush_operations': 0, # u'wr_operations': 0, u'wr_bytes': 0} # } # }, # {u'device': u'drive-ide0-1-0', # u'stats': {u'rd_operations': 0, u'flush_total_time_ns': 0, u'wr_highest_offset': 0, u'rd_total_time_ns': 0, # u'rd_bytes': 0, u'wr_total_time_ns': 0, u'flush_operations': 0, u'wr_operations': 0, u'wr_bytes': 0} # }], # u'id': u'libvirt-9'} stats = {} self._log.debug('#### print device list: %s', devList) for item in devList['return']: fullDevName = item['device'] alias = fullDevName[len('drive-'):].strip() devStats = item['stats'] stats[alias] = {'rd_op':devStats['rd_operations'], 'wr_op':devStats['wr_operations'], 'flush_op':devStats['flush_operations'], 'rd_total_time_ns':devStats['rd_total_time_ns'], 'wr_total_time_ns':devStats['wr_total_time_ns'], 'flush_total_time_ns':devStats['flush_total_time_ns']} ------------------------------------------------------------------------------ from printing I added, i noticed that devList is missing some params function relies on; hence failing on keyError: devList['return'] printing: {u'wr_highest_offset': 3183537664, u'rd_bytes': 70507520, u'wr_operations': 189, u'rd_operations': 3402, u'wr_bytes': 1369088} git commit: 82969b3b5ca2f9487ea56485fafd843b21d4f447
Haim, which libvirt and qemu version are you using? Is there a sane libvirt API to obtain this data?
(In reply to comment #1) > Haim, which libvirt and qemu version are you using? > > Is there a sane libvirt API to obtain this data? well, for libvirt i use latest upstream git; 98b942c6f51f0b383f9438fe1ec678b6e8c4984e from Dec 13. for qemu, qemu-kvm-0.15.1-3.fc16.x86_64 (latest from koji). what kind of API are you considering ?
I think the API you're looking for is called virDomainBlockStatsFlags.
Please check it with http://gerrit.ovirt.org/#/c/4304/