Bug 1374795

Phil, several things to fix in the procedures you have:

- please don't modify /etc/neutron/dnsmasq-neutron.conf, instead completely remove the MTU hack from there: DHCP agent should already advertise the right value to your instances (and the right value for vxlan networks is not 8900 but 8950, so please stick to neutron calculation).

- OVS agent does not load its configuration options from ml2_conf.ini. The config file it reads is /etc/neutron/plugins/ml2/openvswitch_agent.ini. Make sure you configure the agent firewall using the right configuration file: http://docs.openstack.org/draft/networking-guide/config-ovsfwdriver.html

Also, MTU on tap devices is set by nova, not neutron. Please attach nova.conf, and nova-compute service logs.

Hi Ihar,

(In reply to Ihar Hrachyshka from comment #1)
> Phil, several things to fix in the procedures you have:
> 
> - please don't modify /etc/neutron/dnsmasq-neutron.conf, instead completely
> remove the MTU hack from there: DHCP agent should already advertise the
> right value to your instances (and the right value for vxlan networks is not
> 8900 but 8950, so please stick to neutron calculation).
> 
> - OVS agent does not load its configuration options from ml2_conf.ini. The
> config file it reads is /etc/neutron/plugins/ml2/openvswitch_agent.ini. Make
> sure you configure the agent firewall using the right configuration file:
> http://docs.openstack.org/draft/networking-guide/config-ovsfwdriver.html
> 
> Also, MTU on tap devices is set by nova, not neutron. Please attach
> nova.conf, and nova-compute service logs.

Thanks for your instructions! After adjusting my scripts, I can successfully send jumbo frames between instances when using native OVS firewall driver.

Since my problem is resolved and this is not a software bug, I'm closing the ticket now.

Thanks, Phil

Hello team,

Any update on the above concern? Kindly let us know if the above is possible on OSP 10?

This bug is for RDO not OSP. I will repurpose it for OSP right now. I think it makes sense for br-int to have a large MTU that would accommodate for all ports plugged. We will need to have a look how to handle MTU for the new driver.

Note that full support for the driver is, for what I understand, not OSP10. We will look nevertheless.

Jakub, do you have an idea how MTU is supposed to be handled for the new firewall driver? Can we set MTU per port created without enforcing the same value for br-int? Or can we just set a insanely large value of mtu for the bridge like 10000? Should tap MTU even influence traffic flow if traffic is steered with flows?

(In reply to Ihar Hrachyshka from comment #8)
> Jakub, do you have an idea how MTU is supposed to be handled for the new
> firewall driver?

I don't know for sure. I would expect nova to handle it on virt driver level. I looked at os-vif and nova code and they don't set any mtu for non-hybrid plugging, the only thing it does is creating the ovs bridge if it doesn't exist. Then in case of libvirt virt driver tap device is plugged to the bridge according domain XML description. The firewall driver doesn't touch network interface settings.

> Can we set MTU per port created without enforcing the same
> value for br-int?

I think that would be a question for openvswitch team but the br-int interface is just another port in br-int bridge e.g. here I tried to create br-mtu and port-mtu:
    Bridge br-mtu
        Port port-mtu
            Interface port-mtu
                type: internal
        Port br-mtu
            Interface br-mtu
                type: internal

so I don't know if setting lower mtu on port br-mtu can influence the traffic over the bridge.

> Or can we just set a insanely large value of mtu for the
> bridge like 10000?

It doesn't seem like we need high mtu on the port br-int but we need confirmation from ovs team.

> Should tap MTU even influence traffic flow if traffic is
> steered with flows?

No, I don't think it can. In case of traffic coming to VM, the traffic is filtered before it reaches the port so the port MTU is not relevant for flows.

I did a local test where I have two port with mtu 9000 in namespaces and they are interconnected via bridge with mtu set to 1500.

Bridge:
[root@devstack ~]# ip a s br-mtu
1395: br-mtu: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 0e:b6:af:f5:57:43 brd ff:ff:ff:ff:ff:ff

ports:
[root@devstack ~]# ip net e portA ip a s port-mtu
1396: port-mtu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:9f:96:41:9b:aa brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.2/24 scope global port-mtu
       valid_lft forever preferred_lft forever
    inet6 fe80::f89f:96ff:fe41:9baa/64 scope link
       valid_lft forever preferred_lft forever
[root@devstack ~]# ip net e portB ip a s portB
1397: portB: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 42:c9:47:92:f1:b4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/24 scope global portB
       valid_lft forever preferred_lft forever
    inet6 fe80::40c9:47ff:fe92:f1b4/64 scope link
       valid_lft forever preferred_lft forever


I'm sending packets from portB to port-mtu and they pass:
[root@devstack ~]# ip net e portB ping -s 8972 -c1 -M do 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8972(9000) bytes of data.
8980 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.504 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.504/0.504/0.504/0.000 ms

on port-mtu:
[root@devstack ~]# ip net e portA tcpdump -s0 -e -nvvi port-mtu
tcpdump: listening on port-mtu, link-type EN10MB (Ethernet), capture size 262144 bytes
11:47:56.991159 42:c9:47:92:f1:b4 > fa:9f:96:41:9b:aa, ethertype IPv4 (0x0800), length 9014: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 9000)
    192.168.0.1 > 192.168.0.2: ICMP echo request, id 26688, seq 1, length 8980
11:47:56.991210 fa:9f:96:41:9b:aa > 42:c9:47:92:f1:b4, ethertype IPv4 (0x0800), length 9014: (tos 0x0, ttl 64, id 22347, offset 0, flags [none], proto ICMP (1), length 9000)
    192.168.0.2 > 192.168.0.1: ICMP echo reply, id 26688, seq 1, length 8980

I also talked with ovs developer and he said setting mtu on bridge internal port is specific to that port.

Hence br-int mtu shouldn't affect any tenant traffic even with jumbo frames.

According https://libvirt.org/formatdomain.html#mtu the interface mtu configuration can be given by domain XML. I think this is where we should define the mtu. If we don't get correct mtu on instance tap interface then this bug should be flipped to nova.