Bug 1273837 - [SR-IOV] - vdsm.log is spammed with KeyError: 'net.0.name' error messages while running VM with 'pci-passthrough' vNIC/s
[SR-IOV] - vdsm.log is spammed with KeyError: 'net.0.name' error messages whi...
Status: CLOSED CURRENTRELEASE
Product: vdsm
Classification: oVirt
Component: General (Show other bugs)
4.17.0
x86_64 Linux
unspecified Severity medium (vote)
: ovirt-3.6.1
: 4.17.10
Assigned To: Ido Barkan
Meni Yakove
network
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-21 07:26 EDT by Michael Burman
Modified: 2016-02-10 14:16 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-16 07:17:31 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
ylavi: Triaged+
ylavi: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
vdsm log (808.76 KB, application/x-gzip)
2015-10-21 07:26 EDT, Michael Burman
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 47760 master MERGED vmstats: take in account missing bulk stats fields Never
oVirt gerrit 47800 ovirt-3.6 MERGED vmstats: take in account missing bulk stats fields Never

  None (edit)
Description Michael Burman 2015-10-21 07:26:51 EDT
Created attachment 1085092 [details]
vdsm log

Description of problem:
[SR-IOV] - vdsm.log is spammed with KeyError: 'net.0.name' error messages while running VM with 'pci-passthrough' vNIC/s.

Thread-88::ERROR::2015-10-21 14:11:03,494::vm::1390::virt.vm::(_getRunningVmStats) vmId=`cde9bf57-cfd7-4dd2-9ce3-592a75e67f75`::Error fetching vm stats
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 1387, in _getRunningVmStats
    vm_sample.interval)
  File "/usr/share/vdsm/virt/vmstats.py", line 42, in produce
    networks(vm, stats, first_sample, last_sample, interval)
  File "/usr/share/vdsm/virt/vmstats.py", line 213, in networks
    first_indexes = _find_bulk_stats_reverse_map(first_sample, 'net')
  File "/usr/share/vdsm/virt/vmstats.py", line 340, in _find_bulk_stats_reverse_map
    name_to_idx[stats['%s.%d.name' % (group, idx)]] = idx
KeyError: 'net.0.name'

- This error is shown and filled the vdsm.log every minute while the VM is running with 'pci-passthrough' vNIC/s, cause vdsm trying to monitor and get VMstats, but failing to do so.

Version-Release number of selected component (if applicable):
3.6.0.1-0.1.el6.noarch
vdsm-4.17.9-1.el7ev.noarch

Steps to Reproduce:
1. Run VM with VFs - 'pci-passthough' vNIC/s 
2. less /var/log/vdsm/vdsm.log

Actual results:
vdsm.log is spammed with errors.

Expected results:
vdsm.log shouldn't be spammed with such errors.
Comment 1 Ido Barkan 2015-10-27 06:19:36 EDT
When there is a direct pass through device attached to a vm, libvirt cannot report any statistics on it, but still count it in the context of statistics, which made vdsm expect the statistics was there.

Since this is a documented behavior in https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectGetAllDomainStats , vdsm should be defensive and treat this number (in this case "net.count") more as an upper bound.
Comment 2 Yaniv Lavi 2015-10-29 08:49:09 EDT
In oVirt testing is done on single release by default. Therefore I'm removing the 4.0 flag. If you think this bug must be tested in 4.0 as well, please re-add the flag. Please note we might not have testing resources to handle the 4.0 clone.
Comment 3 Sandro Bonazzola 2015-11-24 11:43:26 EST
Please set target release or I can't move the bug to ON_QA automatically.
Comment 4 Red Hat Bugzilla Rules Engine 2015-11-24 13:08:04 EST
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.
Comment 5 Red Hat Bugzilla Rules Engine 2015-11-26 23:38:22 EST
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.
Comment 6 Michael Burman 2015-11-30 01:05:18 EST
Verified on - 3.6.1-0.2.el6 and vdsm-4.17.11-0.el7ev.noarch
Comment 7 Sandro Bonazzola 2015-12-16 07:17:31 EST
According to verification status and target milestone this issue should be fixed in oVirt 3.6.1. Closing current release.

Note You need to log in before you can comment on or make changes to this bug.