Bug 1936610

Summary: setting MTU on VLAN subinterface of bonded interface is ignored
Product: Red Hat Enterprise Linux 8 Reporter: Micah Abbott <miabbott>
Component: NetworkManagerAssignee: Beniamino Galvani <bgalvani>
Status: CLOSED ERRATA QA Contact: Vladimir Benes <vbenes>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.2CC: acardace, atragler, bgalvani, djuran, fge, lrintel, mavazque, miabbott, pgm-rhel-tools, rkhan, sukulkar, till, vbenes
Target Milestone: rcKeywords: Triaged
Target Release: 8.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: NetworkManager-1.32.0-0.1.el8 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 1937723 (view as bug list) Environment:
Last Closed: 2021-11-09 19:29:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1937723    

Description Micah Abbott 2021-03-08 20:00:34 UTC
In BZ#1932502 it was reported that users were having difficulty configuring the MTU on a VLAN subinterface of a bonded interface when using the kernel command line.


https://bugzilla.redhat.com/show_bug.cgi?id=1932502#c9

```
Configuring MTU on the bond with RHCOS 4.7 worked with these kernel args: "bond=bond0:ens3,ens4:mode=802.3ad,lacp_rate=0,miimon=100:9000 ip=192.168.125.10::192.168.125.1:255.255.255.0:testvm.local:bond0:none:9000:"

The next step was trying to setup MTU on a VLAN subinterface for the bond which is the real use case, that failed tho.

These are the kernel args used: "bond=bond0:ens3,ens4:mode=802.3ad,lacp_rate=0,miimon=100:9000 ip=192.168.125.10::192.168.125.1:255.255.255.0:testvm.local:bond0.19:none:9000: vlan=bond0.19:bond0"


We get 1500 MTU configured on bond0 and bond0.19 interfaces.

The system-connections look like this:


/etc/NetworkManager/system-connections/bond0.nmconnection
[connection]
id=bond0
uuid=78fd2db5-22e7-4c80-9993-62285f6c6a96
type=bond
interface-name=bond0
multi-connect=1
permissions=

[bond]
lacp_rate=0
miimon=100
mode=802.3ad

[ipv4]
dns-search=
method=disabled

[ipv6]
addr-gen-mode=eui64
dns-search=
method=disabled

[proxy]

-----------

/etc/NetworkManager/system-connections/bond0.19.nmconnection
[connection]
id=bond0.19
uuid=78fd2db5-22e7-4c80-9993-62285f6c2acf
type=vlan
interface-name=bond0.19
multi-connect=1
permissions=

[ethernet]
mac-address-blacklist=
mtu=9000

[vlan]
egress-priority-map=
flags=1
id=19
ingress-priority-map=
parent=bond0

[ipv4]
address1=192.168.125.10/24,192.168.125.1
dhcp-hostname=testvm.local
dns-search=
may-fail=false
method=manual

[ipv6]
addr-gen-mode=eui64
dhcp-hostname=testvm.local
dns-search=
method=disabled

[proxy]



Versions tested: 
- nmcli tool: 1.26.0-12.1.rhaos4.7.el8
- NetworkManager: 1.26.0-12.1.rhaos4.7.el8.x86_64
```

Beniamino agreed with the assessment and made a PR upstream to fix the problem:

https://bugzilla.redhat.com/show_bug.cgi?id=1932502#c10

```
> The next step was trying to setup MTU on a VLAN subinterface for the bond
> which is the real use case, that failed tho.
> 
> These are the kernel args used:
> "bond=bond0:ens3,ens4:mode=802.3ad,lacp_rate=0,miimon=100:9000
> ip=192.168.125.10::192.168.125.1:255.255.255.0:testvm.local:bond0.19:none:
> 9000: vlan=bond0.19:bond0"
> 
> 
> We get 1500 MTU configured on bond0 and bond0.19 interfaces.

You are right, this command line doesn't work properly because the MTU specified in

  bond=<bondname>[:<bondslaves>:[:<options>[:<mtu>]]]

gets wrongly assigned to the bond ports, instead of the bond itself. I opened a merge request [1] to fix this issue.

For now you can use the ip= argument to specified the MTU for the bond. Just add:

  ip=bond0:none:9000

to the command line.

[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/767
```

We'd like to see this fixed in RHEL 8.4

Comment 1 Gris Ge 2021-03-09 07:51:54 UTC
Hi Micah,

Can you try rpm in https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=35328100 ?

Meanwhile, could you provide business justification for me to request exception in RHEL 8.4.

Comment 2 Micah Abbott 2021-03-09 16:22:28 UTC
(In reply to Gris Ge from comment #1)
> Hi Micah,
> 
> Can you try rpm in
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=35328100 ?

Will do.

> Meanwhile, could you provide business justification for me to request
> exception in RHEL 8.4.

I thought there was usually a template to fill out, but I'll do my best without one.

Users that are trying to perform complex network configurations involving bonded interfaces and VLANs are unable to successfully configure the MTU on VLAN subinterfaces using only the `bond=` parameters.  This causes confusion and difficulty in correctly configuring their networking stack, especially for platforms like RHCOS which require networking be configured early in the boot process in order to successfully install the OS.  The workaround provided for using the `ip=` parameter to set the MTU on the VLAN subinterface is not as intuitive as using the `bond=` parameter, thus I believe it would be beneficial to users to have this fixed.

Comment 3 Micah Abbott 2021-03-10 14:23:50 UTC
It looks like that RPM worked for me:

```
[core@staticvlanbond ~]$ rpm-ostree status
State: idle
Deployments:         
* ostree://da99df2a3fc09eb8de45c071cdd0f53b5680f11214531c699acc445cf0f02f20
                   Version: 48.83.202103091738-0 (2021-03-09T17:41:11Z)

[core@staticvlanbond ~]$ rpm -q NetworkManager
NetworkManager-1.30.0-2.1.rh1936610.el8.x86_64

[core@staticvlanbond ~]$ cat /proc/cmdline 
random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu  ostree=/ostree/boot.1/rhcos/b9edf8495282325ea170ed71c7da76206a588600c9e3987a4995a2047e9d7afd/0 ignition.firstboot ipv6.disable=1 rd.neednet=1 bond=bond0:ens2,ens3:mode=active-backup,miimon=100:9000 ip=10.10.10.10::10.10.10.1:255.255.255.0:staticvlanbond:bond0.100:none:9000: vlan=bond0.100:bond0

[core@staticvlanbond ~]$ nmcli con show
NAME       UUID                                  TYPE      DEVICE    
bond0.100  7f2a8c42-87ea-4dc4-a138-42fb41f637ce  vlan      bond0.100 
bond0      964d49db-dd7f-4219-b6dd-6e5022ef096d  bond      bond0     
ens2       740d2ab2-bffb-41d3-ac86-1e3871cbca86  ethernet  ens2      
ens3       1b81cbe6-c233-4edc-b1f5-88fce29bebb9  ethernet  ens3      

[core@staticvlanbond ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
3: ens3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
7: bond0.100@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:49:83:a2 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.10/24 brd 10.10.10.255 scope global noprefixroute bond0.100
       valid_lft forever preferred_lft forever

[core@staticvlanbond ~]$ sudo cat /etc/NetworkManager/system-connections/*
[connection]       
id=bond0.100                               
uuid=7f2a8c42-87ea-4dc4-a138-42fb41f637ce
type=vlan   
autoconnect-retries=1
interface-name=bond0.100 
multi-connect=1
permissions=
                                                                               
[ethernet]  
mac-address-blacklist=
mtu=9000                                 
                                                                               
[vlan]               
egress-priority-map=
flags=1                                    
id=100         
ingress-priority-map=
parent=bond0   
                                                                               
[ipv4]
address1=10.10.10.10/24,10.10.10.1
dhcp-hostname=staticvlanbond
dhcp-timeout=90          
dns-search=
may-fail=false
method=manual

[ipv6]
addr-gen-mode=eui64
dhcp-hostname=staticvlanbond
dhcp-timeout=90
dns-search=
method=disabled

[proxy]
[connection]
id=bond0
uuid=964d49db-dd7f-4219-b6dd-6e5022ef096d
type=bond
autoconnect-retries=1
interface-name=bond0
multi-connect=1
permissions=

[ethernet]
mac-address-blacklist=
mtu=9000

[bond]
miimon=100
mode=active-backup

[ipv4]
dhcp-timeout=90
dns-search=
method=disabled

[ipv6]
addr-gen-mode=eui64
dhcp-timeout=90
dns-search=
method=disabled

[proxy]
[connection]
id=ens2
uuid=740d2ab2-bffb-41d3-ac86-1e3871cbca86
type=ethernet
autoconnect-retries=1
interface-name=ens2
master=964d49db-dd7f-4219-b6dd-6e5022ef096d
multi-connect=1
permissions=
slave-type=bond
wait-device-timeout=60000

[ethernet]
mac-address-blacklist=
[connection]
id=ens3
uuid=1b81cbe6-c233-4edc-b1f5-88fce29bebb9
type=ethernet
autoconnect-retries=1
interface-name=ens3
master=964d49db-dd7f-4219-b6dd-6e5022ef096d
multi-connect=1
permissions=
slave-type=bond
wait-device-timeout=60000

[ethernet]
mac-address-blacklist=
```

Comment 4 Till Maas 2021-03-17 16:11:16 UTC
*** Bug 1937842 has been marked as a duplicate of this bug. ***

Comment 8 Vladimir Benes 2021-04-29 08:30:09 UTC
A test was added to the unit test suite.

Comment 10 errata-xmlrpc 2021-11-09 19:29:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: NetworkManager security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4361