Description of problem: Hi, With OCP 4.3, we have no way to specify the MTU of the neutron network that is generated by terraform. Neutron allows administrators to generate networks with the --mtu parameter. It seems that the OpenShift installer either doesn't set this value (and uses the OSP default) or uses the maximum MTU (somehow). In either case, administrator need to be able to provide the MTU for the neutron network in install-config.yaml overcloud) [root@openshift-bootstrap-vm ~]# neutron net-show 302e4ce7-19ab-49df-94d1-68854ca12eb5 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead: ~~~ +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | nova | | created_at | 2020-02-06T09:55:32Z | | description | | | id | 302e4ce7-19ab-49df-94d1-68854ca12eb5 | | ipv4_address_scope | | | ipv6_address_scope | | | l2_adjacency | True | | mtu | 8950 | | name | cluster02-ctjd4-openshift | | port_security_enabled | True | | project_id | 218d8aea39b54baabc3726ebabeaa22d | | provider:network_type | vxlan | | provider:physical_network | | | provider:segmentation_id | 10103 | | qos_policy_id | | | revision_number | 5 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | 2f5015a9-3c94-49e4-af0d-b62feebec13c | | tags | openshiftClusterID=cluster02-ctjd4 | | tenant_id | 218d8aea39b54baabc3726ebabeaa22d | | updated_at | 2020-02-06T09:55:38Z | +---------------------------+--------------------------------------+ ~~~ This is critical when there are MTU mismatches elsewhere in the environment. In the attached customer case, we needed to work around this issue (that's outside of the realm of control of OpenShift) by running the following just before the nova instances spawn: ~~~ openstack network set 302e4ce7-19ab-49df-94d1-68854ca12eb5 --mtu=1450 ~~~ It doesn't matter if the problem lies within OpenStack or within the physical network: the OpenShift installer should/must provide and option so that we can configure the MTU of this network. Version-Release number of the following components: rpm -q openshift-ansible rpm -q ansible ansible --version How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
We see from https://github.com/openshift/installer/blob/b87ca03305176def4bd0443ec1be96e01972d1ac/pkg/types/openstack/platform.go That in OCP 4.3, 'externalDNS' was added to the configuration. We should add a parameter, 'networkMTU' (or whatever better naming) and request a specific MTU when creating the internal network. Otherwise, at the customer site, we need to apply the following cumbersome workaround at the moment: ~~~ # Start OCP IPI installation and wait until networks are created and bootstrap node is up # the OCP network now spawns with the max MTU in the cluster, but due to MTU mismatch issues in the path, traffic is # being blackholed openstack network set --mtu 1450 <openshift internal network> Go to bootstrap node console in horizon and send <CTRL-ALT-DEL> ~~~ - Andreas
The OpenShift IPI is generally against adding extra configuration options. Adding more individual options is generally not going to be well received by the OpenShift project. Currently, the only options are going full UPI or changing this option using Neutron manually on the fly. But we are looking at implementing bring-your-own networking (a feature already available in the AWS provider). That is a hybrid between IPI and UPI where you prepare your networking the way you want, and then have the installer create the servers and set up everything else in the networks. We propose to use this "Customer-provisioned Network & Subnets" feature for any specific Neutron configurations rather than adding more install-config knobs one by one.
Andreas many thanks for the detailed feedback. As Tomas explains, this problem will be solved by a future feature where you will be able to create your networking in OpenStack and then instruct the installer to use the existing networking resources while still installing with the IPI workflow. The upcoming UPI installation for OSP will also allow this.
Closing; the linked feature is expected to supersede this request.