Description of problem:
VM performance is increased greatly by enabling jumbo frames, especially when using tunneling. In order to enable jumbo frames, several settings need to be changed, and we don't provide a way to set all of these options at deployment time.

Version-Release number of selected component (if applicable):
OSP 7 GA

How reproducible:
100%

Steps to Reproduce:
1. Enable jumbo frames on switches
2. Configure NIC templates to set jumbo MTU on physical interfaces
3. Deploy OSP 7

Actual results:
VMs are still set to use a default MTU of 1400, no matter what the actual capacity of the physical connections.

Expected results:
Since the performance is greatly enhanced when using jumbo frames, we should enable a way to set this by default for VMs that doesn't involve editing configuration files and restarting Neutron DHCP agents.

Additional info:
Here are all the things that need to be set:

overcloud-without-mergepy.yaml has:

parameters:
  NeutronDnsmasqOptions:
    default: 'dhcp-option-force=26,1400'
    description: Dnsmasq options for neutron-dhcp-agent. The default value here forces MTU to be set to 1400 to account for the tunnel overhead.


So we need to override the default parameter in the Heat stack:

parameter_defaults:
    NeutronDnsmasqOptions: 'dhcp-option-force=26,<MTU>'

That setting gets written to /etc/neutron/dnsmasq-neutron.conf on the controllers.

Then, every compute and controller node has to have these settings:

/etc/nova/nova.conf:

network_device_mtu=<MTU>

/etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini:

veth_mtu = <MTU>

(where <MTU> is the physical MTU - 50 bytes, so if the physical interfaces are set to 9000, this should be set to 8950)

If we had a single command-line parameter, say --default-vm-mtu, then we could set this at deployment time. Alternatively, we could set a Heat parameter for this value, and then use that to create the proper configuration in those three places. Another approach would be to leave the current NeutronDnsmasqOptions parameter as-is, but then add a parameter that sets network_device_mtu and veth_mtu, but then we will need to set both parameters at deployment time.