Bug 1497396

Summary: suggest improve the error message and document for domifstat, domifaddr commands
Product: Red Hat Enterprise Linux 7 Reporter: yalzhang <yalzhang>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Jing Qi <jinqi>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.4CC: dyuan, jishao, mprivozn, rbalakri, xuzhang, yalzhang
Target Milestone: rcKeywords: Upstream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-3.9.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 10:57:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description yalzhang@redhat.com 2017-09-30 05:44:38 UTC
Description of problem:
suggest improve the error message and document for domifstat, domifaddr commands

Version-Release number of selected component (if applicable):
libvirt-3.7.0-2.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.  start a guest with user type interface
# virsh start rhel7
# virsh dumpxml rhel7 | grep /interface -B8
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <interface type='user'>
      <mac address='52:54:00:29:ae:b7'/>
      <target dev='user_target'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:e8:83:93'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='rtl8139'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>

2. on host
# virsh domiflist rhel7 
Interface  Type       Source     Model       MAC
-------------------------------------------------------
user_target user       -          virtio      52:54:00:29:ae:b7
vnet0      bridge     br0        rtl8139     52:54:00:e8:83:93

# virsh domifstat rhel7 user_target 
error: Failed to get interface stats rhel7 user_target
error: internal error: /proc/net/dev: Interface not found


Actual results:
In step 2, domifstat tried to get the user type interface's status but failed, the error message can be improved

Expected results:
domifstat only works for the interface type which need a backend tap on host, such as network type with nat/route/isolated/open network and bridge type, direct type, suggest to improve the error message in step 2 for all other unsupported types.

suggest to improve the document to explain that domifstat only works for specific interface type.

Additional info:
one more  problem, in the man page for domifstat item, no introduction for the option "interface-device", then we can refer to the one below, but it is not appropriate.
Mac address is not accepted by domifstat.

# man virsh
...
 domifstat domain interface-device
           Get network interface stats for a running domain.

domif-setlink domain interface-device state [--config]
           Modify link state of the domain's virtual interface. Possible values for state are "up" and "down". If --config is specified, only the persistent configuration of the
           domain is modified, for compatibility purposes, --persistent is alias of --config.  interface-device can be the interface's target name or the MAC address.

# virsh domiflist rhel7
Interface  Type       Source     Model       MAC
-------------------------------------------------------
vnet0      bridge     ovsbr0     virtio      52:54:00:e8:83:93
vnet1      network    default    rtl8139     52:54:00:d6:4a:cc

# virsh domifstat rhel7 52:54:00:d6:4a:cc
error: Failed to get interface stats rhel7 52:54:00:d6:4a:cc
error: invalid argument: invalid path, '52:54:00:d6:4a:cc' is not a known interface

# virsh domifstat rhel7 fe:54:00:d6:4a:cc
error: Failed to get interface stats rhel7 fe:54:00:d6:4a:cc
error: invalid argument: invalid path, 'fe:54:00:d6:4a:cc' is not a known interface

Comment 2 Michal Privoznik 2017-10-05 14:19:41 UTC
Patches proposed on the upstream list:

https://www.redhat.com/archives/libvir-list/2017-October/msg00246.html

Comment 3 Jing Qi 2017-10-10 06:59:59 UTC
I built the libvirt rpms with upstream code with patches applied. 
With part of xml file as below:

<interface type='network'>
      <mac address='52:54:00:2e:d4:37'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x11' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:2e:d4:37'/>
      <source bridge='br0'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

I did a test, but it has some issues.

# virsh domif-getlink avocado-vt-vm1 52:54:00:2e:d4:37
error: multiple matching interfaces found
# virsh domifstat avocado-vt-vm1 52:54:00:2e:d4:37error: Failed to get interface stats avocado-vt-vm1 52:54:00:2e:d4:37
error: internal error: /proc/net/dev: Interface not found

# virsh domifstat avocado-vt-vm1 vnet0
vnet0 rx_bytes 5952
vnet0 rx_packets 110
vnet0 rx_errs 0
vnet0 rx_drop 0
vnet0 tx_bytes 0
vnet0 tx_packets 0
vnet0 tx_errs 0
vnet0 tx_drop 0

I also tried to use xml file with below part removed. The MAC address didn't work either.
 <interface type='bridge'>
      <mac address='52:54:00:2e:d4:37'/>
      <source bridge='br0'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>


# virsh domifstat avocado-vt-vm1 52:54:00:2e:d4:37error: Failed to get interface stats avocado-vt-vm1 52:54:00:2e:d4:37
error: internal error: /proc/net/dev: Interface not found

Comment 4 Jing Qi 2017-10-10 07:47:05 UTC
More info about last comment 3.

#virsh dumpxml avocado-vt-vm1 |grep -A8 interface
    <interface type='network'>
      <mac address='52:54:00:2e:d4:37'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x11' function='0x0'/>
    </interface>

 
The test of no device is done also-

# virsh domifstat avocado-vt-vm1 vnet9
error: Failed to get interface stats avocado-vt-vm1 vnet9
error: invalid argument: 'vnet9' is not a known interface

Comment 5 Michal Privoznik 2017-10-14 00:49:35 UTC
I've proposed a patch for that on the mailing list:

https://www.redhat.com/archives/libvir-list/2017-October/msg00609.html

Comment 6 Michal Privoznik 2017-10-14 00:57:17 UTC
Aaand I've just pushed the patches:

commit be9978bb89a8b390f457706ff425cc20ff3937a0
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Oct 13 16:29:36 2017 -0700
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 13 17:54:50 2017 -0700

    virDomainInterfaceStats: Accept MAC addresses properly
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1497396
    
    In 0d3d020ba6c4f I've added capability to accept MAC addresses
    for the API too. However, the implementation was faulty. It needs
    to lookup the corresponding interface in the domain definition
    and pass the ifname instead of MAC address.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 7a995c6b78062be85ae03b74511e2cc7f0a08898
Author:     Michal Privoznik <mprivozn>
AuthorDate: Wed Oct 4 17:39:59 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 13 13:47:17 2017 -0700

    virsh: Deal with multiple matching devices in domif-getlink
    
    The command tries to match interface in domain definition by MAC
    address or interface name. However, since it's possible to
    configure two interfaces with the same MAC address, it may
    happen that the XPath returns two or more nodes. We should check
    for that.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 0d3d020ba6c4f4f84cc20821dbd8b542ac994d37
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Oct 13 13:25:42 2017 -0700
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 13 13:46:59 2017 -0700

    virDomainInterfaceStats: Accept MAC address too
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1497396
    
    The other APIs accept both, ifname and MAC address. There's no
    reason virDomainInterfaceStats can't do the same.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit e3909729d277c313f6093b5ff78594d8107721d2
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Oct 13 13:40:47 2017 -0700
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 13 13:46:36 2017 -0700

    virDomainNetFind: Report error if no device found
    
    Every caller reports the error themselves. Might as well move it
    into the function and thus unify it.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 24796f2710f48d93ab360b950d28b121876975d5
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Oct 13 13:15:13 2017 -0700
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 13 13:46:01 2017 -0700

    virsh: Document limitation of domifstat
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1497396
    
    The current implementation reads the stats from the host.
    However, this doesn't work for all types of interfaces as not all
    of them have a representation in the host. For instance,
    interface type='user' doesn't.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>


v3.8.0-88-gbe9978bb8

Comment 7 Jing Qi 2017-10-17 02:12:27 UTC
Build rpm with the patch newly provided and verified it with below steps:

1. Start a domain with two  network interfaces.
[root@hp-dl585g7-01 x86_64]# virsh domiflist avocado-vt-vm1
Interface  Type       Source     Model       MAC
-------------------------------------------------------
vnet0      network    default    virtio      52:54:00:2e:d4:37
user_target user       -          virtio      52:54:00:2e:d4:47

[root@hp-dl585g7-01 x86_64]# virsh domifstat avocado-vt-vm1 52:54:00:2e:d4:37
52:54:00:2e:d4:37 rx_bytes 1844
52:54:00:2e:d4:37 rx_packets 31
52:54:00:2e:d4:37 rx_errs 0
52:54:00:2e:d4:37 rx_drop 0
52:54:00:2e:d4:37 tx_bytes 0
52:54:00:2e:d4:37 tx_packets 0
52:54:00:2e:d4:37 tx_errs 0
52:54:00:2e:d4:37 tx_drop 0

[root@hp-dl585g7-01 x86_64]# virsh domifstat avocado-vt-vm1 vnet0
vnet0 rx_bytes 2156
vnet0 rx_packets 37
vnet0 rx_errs 0
vnet0 rx_drop 0
vnet0 tx_bytes 0
vnet0 tx_packets 0
vnet0 tx_errs 0
vnet0 tx_drop 0



2.Start a domain with three network interfaces and two interfaces have the same MAC address.

# virsh domiflist avocado-vt-vm1
Interface  Type       Source     Model       MAC
-------------------------------------------------------
vnet0      network    default    virtio      52:54:00:2e:d4:37
vnet1      bridge     br0        rtl8139     52:54:00:2e:d4:37
user_target user       -          virtio      52:54:00:2e:d4:47

# virsh domifstat avocado-vt-vm1  vnet1
vnet1 rx_bytes 648
vnet1 rx_packets 8
vnet1 rx_errs 0
vnet1 rx_drop 0
vnet1 tx_bytes 1992
vnet1 tx_packets 12
vnet1 tx_errs 0
vnet1 tx_drop 0

[root@hp-dl585g7-01 x86_64]# virsh domifstat avocado-vt-vm1  vnet0
vnet0 rx_bytes 1688
vnet0 rx_packets 28
vnet0 rx_errs 0
vnet0 rx_drop 0
vnet0 tx_bytes 0
vnet0 tx_packets 0
vnet0 tx_errs 0
vnet0 tx_drop 0
# virsh domifstat avocado-vt-vm1  vnet1
vnet1 rx_bytes 648
vnet1 rx_packets 8
vnet1 rx_errs 0
vnet1 rx_drop 0
vnet1 tx_bytes 11670
vnet1 tx_packets 65
vnet1 tx_errs 0
vnet1 tx_drop 0

[root@hp-dl585g7-01 x86_64]# virsh domifstat avocado-vt-vm1  52:54:00:2e:d4:37
52:54:00:2e:d4:37 rx_bytes 1896
52:54:00:2e:d4:37 rx_packets 32
52:54:00:2e:d4:37 rx_errs 0
52:54:00:2e:d4:37 rx_drop 0
52:54:00:2e:d4:37 tx_bytes 0
52:54:00:2e:d4:37 tx_packets 0
52:54:00:2e:d4:37 tx_errs 0
52:54:00:2e:d4:37 tx_drop 0

Currently, the MAC address can be accepted. And there is one question -
There are only one set of information listed with the MAC address which two interfaces are using it. Is it expected?

Comment 8 Michal Privoznik 2017-10-19 08:15:09 UTC
(In reply to Jing Qi from comment #7)
> Build rpm with the patch newly provided and verified it with below steps:


> Currently, the MAC address can be accepted. And there is one question -
> There are only one set of information listed with the MAC address which two
> interfaces are using it. Is it expected?

Yes. Two MAC addresses are not supported really.

Comment 9 yalzhang@redhat.com 2017-10-24 11:26:19 UTC
Hi Michal, I have all the virsh commands using "interface-device" with duplicate mac, the result as below:

domif-getlink will return error as 'multiple matching interfaces found'
domif-setlink, domifstat, domiftune will operate at the first matching interface. 

Please help to confirm, Thank you!


Start a guest with 2 network interface share the same mac
# virsh domiflist rhel7
Interface  Type       Source     Model       MAC
-------------------------------------------------------
vnet0      network    default    virtio      52:54:00:11:0c:ff
vnet1      network    open       rtl8139     52:54:00:11:0c:ff


1. For domif-getlink, it will report error with the mac

# virsh domif-getlink rhel7 52:54:00:11:0c:ff
error: multiple matching interfaces found

2. For domif-setlink, it will match the first interface with the mac

# virsh domif-getlink rhel7 vnet0
vnet0 up
# virsh domif-getlink rhel7 vnet1
vnet1 up

# virsh domif-setlink rhel7 52:54:00:11:0c:ff down
Device updated successfully

# virsh domif-getlink rhel7 vnet0
vnet0 down
# virsh domif-getlink rhel7 vnet1
vnet1 up

3. For domifstat, it will also matches the first one with the mac as comment7

4. For domiftune, it will also matches the first one with the mac

# virsh domiftune rhel7 52:54:00:11:0c:ff --inbound 100,200,128 --outbound 50,100,256
# virsh domiftune rhel7 52:54:00:11:0c:ff
inbound.average: 100
inbound.peak   : 200
inbound.burst  : 128
inbound.floor  : 0
outbound.average: 50
outbound.peak  : 100
outbound.burst : 256

# virsh dumpxml rhel7 | grep /interface -B13
    <interface type='network'>
      <mac address='52:54:00:11:0c:ff'/>
      <source network='default' bridge='virbr0'/>
      <bandwidth>
        <inbound average='100' peak='200' burst='128'/>
        <outbound average='50' peak='100' burst='256'/>
      </bandwidth>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <link state='down'/>
      <mtu size='9000'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:11:0c:ff'/>
      <source network='open' bridge='virbr4'/>
      <target dev='vnet1'/>
      <model type='rtl8139'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/>
    </interface>

Comment 10 Michal Privoznik 2017-10-25 12:14:06 UTC
(In reply to yalzhang from comment #9)
> Hi Michal, I have all the virsh commands using "interface-device" with
> duplicate mac, the result as below:
> 
> domif-getlink will return error as 'multiple matching interfaces found'
> domif-setlink, domifstat, domiftune will operate at the first matching
> interface. 

Since we cannot change the APIs (we would break the ABI), there's not much we can do here. I'd say these results are expected then.

Comment 12 Jingjing Shao 2017-11-09 09:45:58 UTC
Test with libvirt-3.9.0-1.el7.x86_64, get the result as comment 7 and comment 9

Also check the virsh manpage get the info as below.

domifstat domain interface-device
           Get network interface stats for a running domain. The network interface stats are only available for interfaces that have a physical source interface. This does not include, for example, a
           'user' interface type since it is a virtual LAN with NAT to the outside world. interface-device can be the interface target by name or MAC address.

So change the bug status to verified

Comment 16 errata-xmlrpc 2018-04-10 10:57:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704