Bug 1055484

Marking high priority since rhev-3.4 feature:
http://www.ovirt.org/Features/Host_Network_QoS

depends on the resolution of this bug.

Copy-paste blooper:

The definition should be:
<network>                                                                   
    <name>vdsm-no-bridge</name>                                           
    <forward mode='passthrough'><interface dev='em1.10'/></forward>  
    <bandwidth>
        <inbound average='1000' peak='5000' burst='1024'/>
        <outbound average='1000' burst='1024'/>
    </bandwidth>
</network>

I've proposed patches upstream:

https://www.redhat.com/archives/libvir-list/2014-January/msg01105.html

Hi Michal, would these patches cause the entire traffic through the interface be shaped? Or just that related to the affected network?

I mean, let's take a network that isn't VLAN-tagged, say on interface em1. There might be other networks using em1, maybe with VLAN tagging (e.g. em1.10). I'm assuming these networks will also be affected by the QoS configuration, even though only the traffic from the non-VLAN network was meant to be shaped.

Would it be possible to use tc in a manner that would only limit the bandwidth of the one network?

(In reply to Lior Vernia from comment #5)
> Hi Michal, would these patches cause the entire traffic through the
> interface be shaped? Or just that related to the affected network?
> 
> I mean, let's take a network that isn't VLAN-tagged, say on interface em1.
> There might be other networks using em1, maybe with VLAN tagging (e.g.
> em1.10). I'm assuming these networks will also be affected by the QoS
> configuration, even though only the traffic from the non-VLAN network was
> meant to be shaped.
> 
> Would it be possible to use tc in a manner that would only limit the
> bandwidth of the one network?

With the patch I've puhed upstream the QoS is not set on the em1 nor em1.10 but the macvtap device that is created when domain boots up, e.g. macvtap0 or macvtap@em1. However, from my discussion with Antoni it seems I've not understood your scenario completely as you guys need QoS to be set on em1.10 actually (referring to example in comment 3). Fortunately, kernel is wise enough so despite fact, that em1.10 is created over em1, QoS can be set on both independently (to some extent). So for instance em1.10 can be shaped while em1 is not (= VLAN is shaped while the rest of the traffic is not). However, if em1 is shaped, then em1.10 is shaped too - indirectly = same scenario as pluging 1Gbit NIC to 100Mbit switch.

Having said that, we need a different patch which I'm working on right now.

My understanding of the <bandwidth> limits set in a <network> is that they are setting a limit on the sum total of all interfaces connected to that network, nothing more and nothing less. In cases where exactly that cannot be accomplished, I think it is better to log an UNSUPPORTED error and fail, thus encouraging the use of some setup that *can* provide exactly what is wanted.

My understanding of your patches last week is that they set the individual bandwidth for each guest interface to the amount specified for the network, so the total bandwidth would be netmax * #-of-guests, which doesn't seem useful to me (you can do the same thing by specifying a bandwidth in a default <portgroup>)

For more verbiage, see the messages I just posted to the thread for your already-pushed patches and for my "log UNSUPPORTED" patch:

https://www.redhat.com/archives/libvir-list/2014-January/msg01441.html
https://www.redhat.com/archives/libvir-list/2014-January/msg01442.html

I've proposed the second round of network hook script upstream:

https://www.redhat.com/archives/libvir-list/2014-February/msg00207.html

and in the meantime I've pushed the following patch upstream, which will at least make sure that everyone knows what is and isn't supported by libvirt in terms of network-wide QoS (the patch also updates the documentation in formatnetwork.html)

commit eafb53fec2436ca39949badb16b27901fe5b6ce2
Author: Laine Stump <laine>
Date:   Fri Jan 24 13:58:05 2014 +0200

    network: disallow <bandwidth>/<mac> for bridged/macvtap/hostdev networks
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1057321
    
    pointed out that we weren't honoring the <bandwidth> element in
    libvirt networks using <forward mode='bridge'/>. In fact, these
    networks are just a method of giving a libvirt network name to an
    existing Linux host bridge on the system, and libvirt doesn't have
    enough information to know where to set such limits. We are working on
    a method of supporting network bandwidths for some specific cases of
    <forward mode='bridge'/>, but currently libvirt doesn't support it. So
    the proper thing to do now is just log an error when someone tries to
    put a <bandwidth> element in that type of network. (It's unclear if we
    will be able to do proper bandwidth limiting for macvtap networks, and
    most definitely we will not be able to support it for hostdev
    networks).

I can reproduce this bug with build libvirt-1.1.1-22.el7.x86_64.

steps:
1. define and start one macvtap network with passthrough mode:
# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 macvtap-passthrough  active     no            yes

# virsh net-dumpxml macvtap-passthrough
<network>
  <name>macvtap-passthrough</name>
  <uuid>4c5abd3c-3564-42a0-844a-c350ca0d33a5</uuid>
  <forward dev='enp0s25' mode='passthrough'>
    <interface dev='enp0s25'/>
  </forward>
  <bandwidth>
    <inbound average='1000' peak='5000' burst='1024'/>
    <outbound average='1000' burst='1024'/>
  </bandwidth>
</network>

2. check with tc, can't get any rules for this device.
# tc -d filter show dev enp0s25
# tc -d class show dev enp0s25
# tc -d class show dev enp0s25 parent ffff:

Setting CondNAK:Design, as there's still no general enough design for libvirt how to implement aggregated bandwidth limitation. Moreover, this looks like a RFE, so I'm setting the Keyword:FutureFeature.

So from all the discussion I had, the design that would be accetable for libvirt is still unclear. I mean, in libvirt we need design that will be not tied to a single use case. Having said that, we have introduced solution, that allows every scenario to be implemented - the network hooks. Moreover, when I was introducing QoS control to libvirt I feared that once somebody will come to me that his scenario is not covered. And it's true because there's just infinite traffic shaping scenarios. So I tend to think this falls out of libvirt scope. And I think that even more once the network hooks are introduced. So, I'm closing this one then.

*** This bug has been marked as a duplicate of bug 1064831 ***