Bug 1793720

Summary: Unable to bring up OVNKubernetes cluster on custom mtu
Product: OpenShift Container Platform Reporter: Anurag saxena <anusaxen>
Component: NetworkingAssignee: Phil Cameron <pcameron>
Networking sub component: ovn-kubernetes QA Contact: Anurag saxena <anusaxen>
Status: CLOSED NOTABUG Docs Contact:
Severity: medium    
Priority: medium CC: danw, dcbw, mifiedle, pcameron
Version: 4.4Flags: anusaxen: needinfo-
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-30 15:17:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Anurag saxena 2020-01-21 22:50:34 UTC
Description of problem:Trying to bring up aws cluster on custom mtu say 1600 but i see it comes on default uplink mtu of 8901.


Version-Release number of selected component (if applicable):4.4.0-0.ci-2020-01-21-134224


How reproducible:Always


Steps to Reproduce:
1.Add mtu in install-config.yml

metadata:
  creationTimestamp: null
  name: qe-anusaxen
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: 10.0.0.0/16
  networkType: OpenShiftSDN
  mtu: 1600
  serviceNetwork:
  - 172.30.0.0/16
platform:
  aws:

2. ./openshift-install create cluster

3. ocnkube-config also reflects 8901 mtu

# oc get cm ovnkube-config -n openshift-ovn-kubernetes -oyaml | grep -i mtu
    mtu="8901"


Actual results: Cluster comes default on 8901 uplink mtu

Expected results: expecting mtu to be 1600

Additional info: Let me know if i am mising something in configuration

Comment 1 Anurag saxena 2020-01-21 23:10:17 UTC
*Typo's

networkType was OVNKUbernetes with that mtu value 
and ocnkube-config=ovnkube-config

Comment 2 Dan Winship 2020-01-22 08:22:21 UTC
> 1. Add mtu in install-config.yml

install-config doesn't allow configuring MTU... you need to create a network operator config, as with https://docs.openshift.com/container-platform/4.2/installing/installing_aws/installing-aws-network-customizations.html#modifying-nwoperator-config-startup_installing-aws-network-customizations, but using "type: OVNKubernetes" and "ovnKubernetesConfig:"

Comment 3 Anurag saxena 2020-01-22 14:00:44 UTC
(In reply to Dan Winship from comment #2)
> > 1. Add mtu in install-config.yml
> 
> install-config doesn't allow configuring MTU... you need to create a network
> operator config, as with
> https://docs.openshift.com/container-platform/4.2/installing/installing_aws/
> installing-aws-network-customizations.html#modifying-nwoperator-config-
> startup_installing-aws-network-customizations, but using "type:
> OVNKubernetes" and "ovnKubernetesConfig:"

Hi Dan, thanks for confirming that. I initially tried it openshift-sdn way like creating cluster-network-03-config.yml that was consumed during create cluster
but did't see it working.

>> cluster-network-03-config.yml

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec: 
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  serviceNetwork:
  - 172.30.0.0/16
  defaultNetwork:
    type: OVNKubernetes
    ovnKubernetesConfig:
      mtu: 1600

But i believe its working for Phil the install-config.yml way as per discussion on Forum-sdn so gave a try to that way too. Let me try creating x-x-03-config.yml and see again. Thanks. I am sure the right way is x-x-03-config.yml but not sure how its working for Phil. Phil, any comments?

Comment 4 Phil Cameron 2020-01-22 14:33:31 UTC
@anurag Please look try
oc logs -c ovnkube-master ovnkube-master-XXXX | grep "Default:{MTU:" 
where XXXX is the pod.
Also
oc get cm ovnkube-config -n openshift-ovn-kubernetes -oyaml | grep "^data:" -A17

Comment 6 Phil Cameron 2020-01-22 19:53:37 UTC
@anurag is this ready to be closed? Anything left to do?

Comment 7 Anurag saxena 2020-01-22 20:08:25 UTC
@Phil, thanks for asking. If comment 5 steps seems okay to you, we can close this. I believe you tested install-config way so just wondering.

Comment 8 Phil Cameron 2020-01-22 21:03:33 UTC
@anurag Steps in comment 5 are fine by me. I think I accidentally picked 9001 that were converted into 8901 and I didn't spot what was really happening. The task is to be able to pick a different (unexpected) mtu at install and see it applied in the cluster. The instructions on how to do that are not part of the requirement. You have shown that it works so that proves it.

Comment 9 Anurag saxena 2020-01-23 15:40:50 UTC
Thanks @Phil for confirming the steps. We can close this bug.

Comment 10 Phil Cameron 2020-01-30 15:17:03 UTC
Not a bug. This works when using the correct procedure for creating the ovn cluster.