Bug 1937723

+++ This bug was initially created as a clone of Bug #1936610 +++

In BZ#1932502 it was reported that users were having difficulty configuring the MTU on a VLAN subinterface of a bonded interface when using the kernel command line.


https://bugzilla.redhat.com/show_bug.cgi?id=1932502#c9

```
Configuring MTU on the bond with RHCOS 4.7 worked with these kernel args: "bond=bond0:ens3,ens4:mode=802.3ad,lacp_rate=0,miimon=100:9000 ip=192.168.125.10::192.168.125.1:255.255.255.0:testvm.local:bond0:none:9000:"

The next step was trying to setup MTU on a VLAN subinterface for the bond which is the real use case, that failed tho.

These are the kernel args used: "bond=bond0:ens3,ens4:mode=802.3ad,lacp_rate=0,miimon=100:9000 ip=192.168.125.10::192.168.125.1:255.255.255.0:testvm.local:bond0.19:none:9000: vlan=bond0.19:bond0"


We get 1500 MTU configured on bond0 and bond0.19 interfaces.

The system-connections look like this:


/etc/NetworkManager/system-connections/bond0.nmconnection
[connection]
id=bond0
uuid=78fd2db5-22e7-4c80-9993-62285f6c6a96
type=bond
interface-name=bond0
multi-connect=1
permissions=

[bond]
lacp_rate=0
miimon=100
mode=802.3ad

[ipv4]
dns-search=
method=disabled

[ipv6]
addr-gen-mode=eui64
dns-search=
method=disabled

[proxy]

-----------

/etc/NetworkManager/system-connections/bond0.19.nmconnection
[connection]
id=bond0.19
uuid=78fd2db5-22e7-4c80-9993-62285f6c2acf
type=vlan
interface-name=bond0.19
multi-connect=1
permissions=

[ethernet]
mac-address-blacklist=
mtu=9000

[vlan]
egress-priority-map=
flags=1
id=19
ingress-priority-map=
parent=bond0

[ipv4]
address1=192.168.125.10/24,192.168.125.1
dhcp-hostname=testvm.local
dns-search=
may-fail=false
method=manual

[ipv6]
addr-gen-mode=eui64
dhcp-hostname=testvm.local
dns-search=
method=disabled

[proxy]



Versions tested: 
- nmcli tool: 1.26.0-12.1.rhaos4.7.el8
- NetworkManager: 1.26.0-12.1.rhaos4.7.el8.x86_64
```

Beniamino agreed with the assessment and made a PR upstream to fix the problem:

https://bugzilla.redhat.com/show_bug.cgi?id=1932502#c10

```
> The next step was trying to setup MTU on a VLAN subinterface for the bond
> which is the real use case, that failed tho.
> 
> These are the kernel args used:
> "bond=bond0:ens3,ens4:mode=802.3ad,lacp_rate=0,miimon=100:9000
> ip=192.168.125.10::192.168.125.1:255.255.255.0:testvm.local:bond0.19:none:
> 9000: vlan=bond0.19:bond0"
> 
> 
> We get 1500 MTU configured on bond0 and bond0.19 interfaces.

You are right, this command line doesn't work properly because the MTU specified in

  bond=<bondname>[:<bondslaves>:[:<options>[:<mtu>]]]

gets wrongly assigned to the bond ports, instead of the bond itself. I opened a merge request [1] to fix this issue.

For now you can use the ip= argument to specified the MTU for the bond. Just add:

  ip=bond0:none:9000

to the command line.

[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/767
```

We'd like to see this fixed in RHEL 8.4

--- Additional comment from Gris Ge on 2021-03-09 15:51:54 CST ---

Hi Micah,

Can you try rpm in https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=35328100 ?

Meanwhile, could you provide business justification for me to request exception in RHEL 8.4.

--- Additional comment from Micah Abbott on 2021-03-10 00:22:28 CST ---

(In reply to Gris Ge from comment #1)
> Hi Micah,
> 
> Can you try rpm in
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=35328100 ?

Will do.

> Meanwhile, could you provide business justification for me to request
> exception in RHEL 8.4.

I thought there was usually a template to fill out, but I'll do my best without one.

Users that are trying to perform complex network configurations involving bonded interfaces and VLANs are unable to successfully configure the MTU on VLAN subinterfaces using only the `bond=` parameters.  This causes confusion and difficulty in correctly configuring their networking stack, especially for platforms like RHCOS which require networking be configured early in the boot process in order to successfully install the OS.  The workaround provided for using the `ip=` parameter to set the MTU on the VLAN subinterface is not as intuitive as using the `bond=` parameter, thus I believe it would be beneficial to users to have this fixed.

--- Additional comment from Micah Abbott on 2021-03-10 22:23:50 CST ---

It looks like that RPM worked for me:

```
[core@staticvlanbond ~]$ rpm-ostree status
State: idle
Deployments:         
* ostree://da99df2a3fc09eb8de45c071cdd0f53b5680f11214531c699acc445cf0f02f20
                   Version: 48.83.202103091738-0 (2021-03-09T17:41:11Z)

[core@staticvlanbond ~]$ rpm -q NetworkManager
NetworkManager-1.30.0-2.1.rh1936610.el8.x86_64

[core@staticvlanbond ~]$ cat /proc/cmdline 
random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu  ostree=/ostree/boot.1/rhcos/b9edf8495282325ea170ed71c7da76206a588600c9e3987a4995a2047e9d7afd/0 ignition.firstboot ipv6.disable=1 rd.neednet=1 bond=bond0:ens2,ens3:mode=active-backup,miimon=100:9000 ip=10.10.10.10::10.10.10.1:255.255.255.0:staticvlanbond:bond0.100:none:9000: vlan=bond0.100:bond0

[core@staticvlanbond ~]$ nmcli con show
NAME       UUID                                  TYPE      DEVICE    
bond0.100  7f2a8c42-87ea-4dc4-a138-42fb41f637ce  vlan      bond0.100 
bond0      964d49db-dd7f-4219-b6dd-6e5022ef096d  bond      bond0     
ens2       740d2ab2-bffb-41d3-ac86-1e3871cbca86  ethernet  ens2      
ens3       1b81cbe6-c233-4edc-b1f5-88fce29bebb9  ethernet  ens3      

[core@staticvlanbond ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
3: ens3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
7: bond0.100@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.10/24 brd 10.10.10.255 scope global noprefixroute bond0.100
       valid_lft forever preferred_lft forever

[core@staticvlanbond ~]$ sudo cat /etc/NetworkManager/system-connections/*
[connection]       
id=bond0.100                               
uuid=7f2a8c42-87ea-4dc4-a138-42fb41f637ce
type=vlan   
autoconnect-retries=1
interface-name=bond0.100 
multi-connect=1
permissions=
                                                                               
[ethernet]  
mac-address-blacklist=
mtu=9000                                 
                                                                               
[vlan]               
egress-priority-map=
flags=1                                    
id=100         
ingress-priority-map=
parent=bond0   
                                                                               
[ipv4]
address1=10.10.10.10/24,10.10.10.1
dhcp-hostname=staticvlanbond
dhcp-timeout=90          
dns-search=
may-fail=false
method=manual

[ipv6]
addr-gen-mode=eui64
dhcp-hostname=staticvlanbond
dhcp-timeout=90
dns-search=
method=disabled

[proxy]
[connection]
id=bond0
uuid=964d49db-dd7f-4219-b6dd-6e5022ef096d
type=bond
autoconnect-retries=1
interface-name=bond0
multi-connect=1
permissions=

[ethernet]
mac-address-blacklist=
mtu=9000

[bond]
miimon=100
mode=active-backup

[ipv4]
dhcp-timeout=90
dns-search=
method=disabled

[ipv6]
addr-gen-mode=eui64
dhcp-timeout=90
dns-search=
method=disabled

[proxy]
[connection]
id=ens2
uuid=740d2ab2-bffb-41d3-ac86-1e3871cbca86
type=ethernet
autoconnect-retries=1
interface-name=ens2
master=964d49db-dd7f-4219-b6dd-6e5022ef096d
multi-connect=1
permissions=
slave-type=bond
wait-device-timeout=60000

[ethernet]
mac-address-blacklist=
[connection]
id=ens3
uuid=1b81cbe6-c233-4edc-b1f5-88fce29bebb9
type=ethernet
autoconnect-retries=1
interface-name=ens3
master=964d49db-dd7f-4219-b6dd-6e5022ef096d
multi-connect=1
permissions=
slave-type=bond
wait-device-timeout=60000

[ethernet]
mac-address-blacklist=
```