Hide Forgot
I have an OpenStack setup with VXLAN tenant networks which is configured to use jumbo frames. There is a dedicated network between all nodes which is configured to an MTU size of 9000. To configure Neutron to use the larger MTU size, the following configuration changes were performed on the network node: # openstack-config --set /etc/neutron/neutron.conf DEFAULT global_physnet_mtu 9000 # openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 path_mtu 9000 # sed -i -e 's/\(dhcp-option-force=26,\)1400/\18900/' \ /etc/neutron/dnsmasq-neutron.conf After recreating the tenant network and all instances, functionality could be verified by sending large pings between instances. After that, I switched to native OVS firewall like so: # openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini \ securitygroup firewall_driver openvswitch Then rebooted all nodes (no idea what services I have to restart to make the config changes get effective) and after they came back up started the instances again. It appears that indeed native OVS firewall is being used since at least QEMU tap devices are directly attached to OVS br-int, but they are configured to MTU 1500 instead of 8900 as I would have expected. Instances on the other hand picked up dhcp-option-26 and have their eth0 interface configured to MTU 8900. After configuring all tap devices manually to MTU 8900, sending large pings between instances works again.
Phil, several things to fix in the procedures you have: - please don't modify /etc/neutron/dnsmasq-neutron.conf, instead completely remove the MTU hack from there: DHCP agent should already advertise the right value to your instances (and the right value for vxlan networks is not 8900 but 8950, so please stick to neutron calculation). - OVS agent does not load its configuration options from ml2_conf.ini. The config file it reads is /etc/neutron/plugins/ml2/openvswitch_agent.ini. Make sure you configure the agent firewall using the right configuration file: http://docs.openstack.org/draft/networking-guide/config-ovsfwdriver.html Also, MTU on tap devices is set by nova, not neutron. Please attach nova.conf, and nova-compute service logs.
Hi Ihar, (In reply to Ihar Hrachyshka from comment #1) > Phil, several things to fix in the procedures you have: > > - please don't modify /etc/neutron/dnsmasq-neutron.conf, instead completely > remove the MTU hack from there: DHCP agent should already advertise the > right value to your instances (and the right value for vxlan networks is not > 8900 but 8950, so please stick to neutron calculation). > > - OVS agent does not load its configuration options from ml2_conf.ini. The > config file it reads is /etc/neutron/plugins/ml2/openvswitch_agent.ini. Make > sure you configure the agent firewall using the right configuration file: > http://docs.openstack.org/draft/networking-guide/config-ovsfwdriver.html > > Also, MTU on tap devices is set by nova, not neutron. Please attach > nova.conf, and nova-compute service logs. Thanks for your instructions! After adjusting my scripts, I can successfully send jumbo frames between instances when using native OVS firewall driver. Since my problem is resolved and this is not a software bug, I'm closing the ticket now. Thanks, Phil
Hello team, Any update on the above concern? Kindly let us know if the above is possible on OSP 10?
This bug is for RDO not OSP. I will repurpose it for OSP right now. I think it makes sense for br-int to have a large MTU that would accommodate for all ports plugged. We will need to have a look how to handle MTU for the new driver. Note that full support for the driver is, for what I understand, not OSP10. We will look nevertheless.
Jakub, do you have an idea how MTU is supposed to be handled for the new firewall driver? Can we set MTU per port created without enforcing the same value for br-int? Or can we just set a insanely large value of mtu for the bridge like 10000? Should tap MTU even influence traffic flow if traffic is steered with flows?
(In reply to Ihar Hrachyshka from comment #8) > Jakub, do you have an idea how MTU is supposed to be handled for the new > firewall driver? I don't know for sure. I would expect nova to handle it on virt driver level. I looked at os-vif and nova code and they don't set any mtu for non-hybrid plugging, the only thing it does is creating the ovs bridge if it doesn't exist. Then in case of libvirt virt driver tap device is plugged to the bridge according domain XML description. The firewall driver doesn't touch network interface settings. > Can we set MTU per port created without enforcing the same > value for br-int? I think that would be a question for openvswitch team but the br-int interface is just another port in br-int bridge e.g. here I tried to create br-mtu and port-mtu: Bridge br-mtu Port port-mtu Interface port-mtu type: internal Port br-mtu Interface br-mtu type: internal so I don't know if setting lower mtu on port br-mtu can influence the traffic over the bridge. > Or can we just set a insanely large value of mtu for the > bridge like 10000? It doesn't seem like we need high mtu on the port br-int but we need confirmation from ovs team. > Should tap MTU even influence traffic flow if traffic is > steered with flows? No, I don't think it can. In case of traffic coming to VM, the traffic is filtered before it reaches the port so the port MTU is not relevant for flows.
I did a local test where I have two port with mtu 9000 in namespaces and they are interconnected via bridge with mtu set to 1500. Bridge: [root@devstack ~]# ip a s br-mtu 1395: br-mtu: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 0e:b6:af:f5:57:43 brd ff:ff:ff:ff:ff:ff ports: [root@devstack ~]# ip net e portA ip a s port-mtu 1396: port-mtu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN qlen 1000 link/ether fa:9f:96:41:9b:aa brd ff:ff:ff:ff:ff:ff inet 192.168.0.2/24 scope global port-mtu valid_lft forever preferred_lft forever inet6 fe80::f89f:96ff:fe41:9baa/64 scope link valid_lft forever preferred_lft forever [root@devstack ~]# ip net e portB ip a s portB 1397: portB: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN qlen 1000 link/ether 42:c9:47:92:f1:b4 brd ff:ff:ff:ff:ff:ff inet 192.168.0.1/24 scope global portB valid_lft forever preferred_lft forever inet6 fe80::40c9:47ff:fe92:f1b4/64 scope link valid_lft forever preferred_lft forever I'm sending packets from portB to port-mtu and they pass: [root@devstack ~]# ip net e portB ping -s 8972 -c1 -M do 192.168.0.2 PING 192.168.0.2 (192.168.0.2) 8972(9000) bytes of data. 8980 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.504 ms --- 192.168.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.504/0.504/0.504/0.000 ms on port-mtu: [root@devstack ~]# ip net e portA tcpdump -s0 -e -nvvi port-mtu tcpdump: listening on port-mtu, link-type EN10MB (Ethernet), capture size 262144 bytes 11:47:56.991159 42:c9:47:92:f1:b4 > fa:9f:96:41:9b:aa, ethertype IPv4 (0x0800), length 9014: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 9000) 192.168.0.1 > 192.168.0.2: ICMP echo request, id 26688, seq 1, length 8980 11:47:56.991210 fa:9f:96:41:9b:aa > 42:c9:47:92:f1:b4, ethertype IPv4 (0x0800), length 9014: (tos 0x0, ttl 64, id 22347, offset 0, flags [none], proto ICMP (1), length 9000) 192.168.0.2 > 192.168.0.1: ICMP echo reply, id 26688, seq 1, length 8980 I also talked with ovs developer and he said setting mtu on bridge internal port is specific to that port. Hence br-int mtu shouldn't affect any tenant traffic even with jumbo frames.
According https://libvirt.org/formatdomain.html#mtu the interface mtu configuration can be given by domain XML. I think this is where we should define the mtu. If we don't get correct mtu on instance tap interface then this bug should be flipped to nova.