Bug 1644941

Summary: Problems enabling jumbo frames
Product: Red Hat OpenStack Reporter: Lars Kellogg-Stedman <lars>
Component: rhosp-directorAssignee: RHOS Maint <rhos-maint>
Status: CLOSED NOTABUG QA Contact: Gurenko Alex <agurenko>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 13.0 (Queens)CC: beagles, bfournie, dbecker, dsneddon, jslagle, jtaleric, mburns, morazi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-12 15:24:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
output of running ip addr on all overcloud hosts none

Description Lars Kellogg-Stedman 2018-11-01 01:37:46 UTC 
Created attachment 1499698 [details]
output of running ip addr on all overcloud hosts

Description of problem:

We are trying to enable jumbo frames for the openstack tenant, storage, and internal api networks.  We have set `NeutronGlobalPhysnetMtu: 9000` in our deploy environments, and in our network configuration we have set an explicit mtu on the bridge devices associated with these networks [1]

[1]: https://github.com/CCI-MOC/rhosp-director-config/tree/new-kaizen/templates/network/config

When the deploy has finished, on the networker:

- `br-ex` and the associated vlan interface both have an MTU of 9000, although this wasn't explicitly set in the network configuration. 

- `br-int` and `br-tun` have an MTU of 1500

On the compute hosts:

- `br-int` and `br-tun` have an MTU of 1500

A complete dump of running `ip addr` on all the hosts in the overcloud is attached to this bz.

Given that we expect tenant network to be using an MTU of 9000, the MTU on br-tun seems incorrect.  On the other hand, we expect br-ex, which is used for floating ips, to have an MTU of 1500.
Comment 1 Lars Kellogg-Stedman 2018-11-01 13:05:02 UTC 
Templates and environment files are all available at:

  https://github.com/CCI-MOC/rhosp-director-config/tree/new-kaizen

The initial deploy command is:

  https://github.com/CCI-MOC/rhosp-director-config/blob/new-kaizen/overcloud-deploy.sh

Network configs are in:

  https://github.com/CCI-MOC/rhosp-director-config/tree/new-kaizen/templates/network/config
Comment 2 Bob Fournier 2018-11-08 20:21:38 UTC 
Including networking DFG
Comment 4 Dan Sneddon 2018-11-08 23:50:01 UTC 
There is a specific setting that allows you to override the global_physnet_mtu for a specific bridge. For example::

parameter_defaults:
  NeutronGlobalPhysnetMtu: 9000
  NeutronML2PhysicalNetworkMtus: [br-ex:1500]


You could even do:

parameter_defaults:
  NeutronML2PhysicalNetworkMtus: [br-ex:1500,br-tenant:9000]

But that said, I don't think the br-int and br-tun MTUs matter in this case. If I'm not mistaken, OVS bridges will actually pass larger frames, and the indicated MTU in "ip addr" only applies if you were to put an IP on br-int or br-tun and generate traffic from that IP, the traffic would be limited to 1500 MTU. Just passing traffic between bridges entirely inside OVS shouldn't be limited to a lower MTU to the best of my knowledge.
Comment 5 Lars Kellogg-Stedman 2018-11-12 14:59:29 UTC 
I've made the MTU explicit on all of our NIC configs.  This seems to have solved the issue with the erroneous MTU on br-ex. Despite the odd-looking MTUs on the OVS bridges, we are able to pass jumbo frames between instances on different networks so I think this is resolved from our perspective.
Comment 6 Bob Fournier 2018-11-12 15:24:57 UTC 
Thanks Dan and Lars.  Closing this out.