Bug 1055484 - bridge-less network creation with QoS element is created without any QoS
bridge-less network creation with QoS element is created without any QoS
Status: CLOSED DUPLICATE of bug 1064831
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.6
Unspecified Unspecified
high Severity unspecified
: rc
: ---
Assigned To: Michal Privoznik
Virtualization Bugs
: FutureFeature
Depends On:
Blocks: 1023565 1002699 1043226
  Show dependency treegraph
 
Reported: 2014-01-20 06:30 EST by Antoni Segura Puimedon
Modified: 2014-12-11 11:13 EST (History)
16 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-04-04 10:27:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Antoni Segura Puimedon 2014-01-20 06:30:44 EST
Description of problem:
According the libvirt documentation, it is possible to create a network specifying QoS attributes ( http://libvirt.org/formatnetwork.html ). Nothing is said about it only applying to bridged networks and indeed libvirt creates networks from such definitions without failure nor warning. However, no TC command is called when the source of the network is not a bridge.

It is very important for the consistency of network creation that when the source is a bond, nic or vlan devices that the traffic control defined by the QoS element is defined as well in the system.


Version-Release number of selected component (if applicable):


How reproducible:100%


Steps to Reproduce:
1. Create a libvirt network definition with QoS:
<network>                                                                   
    <name>vdsm-no-bridge</name>                                           
    <forward mode='passthrough'><interface dev='em1.10'/>     
    <bandwidth>
        <inbound average='1000' peak='5000' burst='1024'/>
        <outbound average='1000' burst='1024'/>
    </bandwidth>
</network>
2. Feed it to libvirt.

Actual results:
No traffic control is applied to the network pass through device.


Expected results:
Traffic control is applied to the network pass through device just as it is
when the forward mode is bridge.


Additional info:
This impacts network Host QoS for oVirt.
Comment 1 Antoni Segura Puimedon 2014-01-20 06:35:00 EST
Marking high priority since rhev-3.4 feature:
http://www.ovirt.org/Features/Host_Network_QoS

depends on the resolution of this bug.
Comment 3 Antoni Segura Puimedon 2014-01-22 08:50:06 EST
Copy-paste blooper:

The definition should be:
<network>                                                                   
    <name>vdsm-no-bridge</name>                                           
    <forward mode='passthrough'><interface dev='em1.10'/></forward>  
    <bandwidth>
        <inbound average='1000' peak='5000' burst='1024'/>
        <outbound average='1000' burst='1024'/>
    </bandwidth>
</network>
Comment 4 Michal Privoznik 2014-01-23 08:51:16 EST
I've proposed patches upstream:

https://www.redhat.com/archives/libvir-list/2014-January/msg01105.html
Comment 5 Lior Vernia 2014-01-26 07:29:16 EST
Hi Michal, would these patches cause the entire traffic through the interface be shaped? Or just that related to the affected network?

I mean, let's take a network that isn't VLAN-tagged, say on interface em1. There might be other networks using em1, maybe with VLAN tagging (e.g. em1.10). I'm assuming these networks will also be affected by the QoS configuration, even though only the traffic from the non-VLAN network was meant to be shaped.

Would it be possible to use tc in a manner that would only limit the bandwidth of the one network?
Comment 6 Michal Privoznik 2014-01-29 08:24:07 EST
(In reply to Lior Vernia from comment #5)
> Hi Michal, would these patches cause the entire traffic through the
> interface be shaped? Or just that related to the affected network?
> 
> I mean, let's take a network that isn't VLAN-tagged, say on interface em1.
> There might be other networks using em1, maybe with VLAN tagging (e.g.
> em1.10). I'm assuming these networks will also be affected by the QoS
> configuration, even though only the traffic from the non-VLAN network was
> meant to be shaped.
> 
> Would it be possible to use tc in a manner that would only limit the
> bandwidth of the one network?

With the patch I've puhed upstream the QoS is not set on the em1 nor em1.10 but the macvtap device that is created when domain boots up, e.g. macvtap0@em1.10 or macvtap@em1. However, from my discussion with Antoni it seems I've not understood your scenario completely as you guys need QoS to be set on em1.10 actually (referring to example in comment 3). Fortunately, kernel is wise enough so despite fact, that em1.10 is created over em1, QoS can be set on both independently (to some extent). So for instance em1.10 can be shaped while em1 is not (= VLAN is shaped while the rest of the traffic is not). However, if em1 is shaped, then em1.10 is shaped too - indirectly = same scenario as pluging 1Gbit NIC to 100Mbit switch.

Having said that, we need a different patch which I'm working on right now.
Comment 7 Laine Stump 2014-01-29 09:57:13 EST
My understanding of the <bandwidth> limits set in a <network> is that they are setting a limit on the sum total of all interfaces connected to that network, nothing more and nothing less. In cases where exactly that cannot be accomplished, I think it is better to log an UNSUPPORTED error and fail, thus encouraging the use of some setup that *can* provide exactly what is wanted.

My understanding of your patches last week is that they set the individual bandwidth for each guest interface to the amount specified for the network, so the total bandwidth would be netmax * #-of-guests, which doesn't seem useful to me (you can do the same thing by specifying a bandwidth in a default <portgroup>)

For more verbiage, see the messages I just posted to the thread for your already-pushed patches and for my "log UNSUPPORTED" patch:

https://www.redhat.com/archives/libvir-list/2014-January/msg01441.html
https://www.redhat.com/archives/libvir-list/2014-January/msg01442.html
Comment 8 Michal Privoznik 2014-02-05 05:15:56 EST
I've proposed the second round of network hook script upstream:

https://www.redhat.com/archives/libvir-list/2014-February/msg00207.html
Comment 9 Laine Stump 2014-02-05 09:10:04 EST
and in the meantime I've pushed the following patch upstream, which will at least make sure that everyone knows what is and isn't supported by libvirt in terms of network-wide QoS (the patch also updates the documentation in formatnetwork.html)

commit eafb53fec2436ca39949badb16b27901fe5b6ce2
Author: Laine Stump <laine@laine.org>
Date:   Fri Jan 24 13:58:05 2014 +0200

    network: disallow <bandwidth>/<mac> for bridged/macvtap/hostdev networks
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1057321
    
    pointed out that we weren't honoring the <bandwidth> element in
    libvirt networks using <forward mode='bridge'/>. In fact, these
    networks are just a method of giving a libvirt network name to an
    existing Linux host bridge on the system, and libvirt doesn't have
    enough information to know where to set such limits. We are working on
    a method of supporting network bandwidths for some specific cases of
    <forward mode='bridge'/>, but currently libvirt doesn't support it. So
    the proper thing to do now is just log an error when someone tries to
    put a <bandwidth> element in that type of network. (It's unclear if we
    will be able to do proper bandwidth limiting for macvtap networks, and
    most definitely we will not be able to support it for hostdev
    networks).
Comment 10 Xuesong Zhang 2014-02-10 03:33:45 EST
I can reproduce this bug with build libvirt-1.1.1-22.el7.x86_64.

steps:
1. define and start one macvtap network with passthrough mode:
# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 macvtap-passthrough  active     no            yes

# virsh net-dumpxml macvtap-passthrough
<network>
  <name>macvtap-passthrough</name>
  <uuid>4c5abd3c-3564-42a0-844a-c350ca0d33a5</uuid>
  <forward dev='enp0s25' mode='passthrough'>
    <interface dev='enp0s25'/>
  </forward>
  <bandwidth>
    <inbound average='1000' peak='5000' burst='1024'/>
    <outbound average='1000' burst='1024'/>
  </bandwidth>
</network>

2. check with tc, can't get any rules for this device.
# tc -d filter show dev enp0s25
# tc -d class show dev enp0s25
# tc -d class show dev enp0s25 parent ffff:
Comment 11 Michal Privoznik 2014-03-12 10:17:01 EDT
Setting CondNAK:Design, as there's still no general enough design for libvirt how to implement aggregated bandwidth limitation. Moreover, this looks like a RFE, so I'm setting the Keyword:FutureFeature.
Comment 13 Michal Privoznik 2014-04-04 10:27:34 EDT
So from all the discussion I had, the design that would be accetable for libvirt is still unclear. I mean, in libvirt we need design that will be not tied to a single use case. Having said that, we have introduced solution, that allows every scenario to be implemented - the network hooks. Moreover, when I was introducing QoS control to libvirt I feared that once somebody will come to me that his scenario is not covered. And it's true because there's just infinite traffic shaping scenarios. So I tend to think this falls out of libvirt scope. And I think that even more once the network hooks are introduced. So, I'm closing this one then.

*** This bug has been marked as a duplicate of bug 1064831 ***

Note You need to log in before you can comment on or make changes to this bug.