Bug 2092762

Summary: Changing vlan-filtering, stp or mtu on bridge detachs unmanaged ports
Product: Red Hat Enterprise Linux 8 Reporter: Quique Llorente <ellorent>
Component: NetworkManagerAssignee: Lubomir Rintel <lrintel>
Status: CLOSED ERRATA QA Contact: Matej Berezny <mberezny>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 8.4CC: acabral, bgalvani, bnemec, ferferna, fge, jiji, jishi, lrintel, mberezny, network-qe, rkhan, sfaye, sukulkar, thaller, till, vbenes
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: NetworkManager-1.39.90-1.el8 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-11-08 10:10:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2092204    

Description Quique Llorente 2022-06-02 08:08:48 UTC
Description of problem:
When a unmanaged interfaces (the veth of a pod at openshift) is a port of managed bridge (one created with kubernetes-nmstate) if the mtu, stp or vlan-filtering on that bridge is changed the veth pot is detacched.



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:

1.
sudo ip link add vethfoo1 type veth peer name vethfoo2

2. 
nmstatectl apply 
interfaces:
  - name: brfoo
    type: linux-bridge
    state: up
    
3. 
sudo ip link set dev vethfoo1 master brfoo

4. 
nmstatectl apply
interfaces:
  - name: brfoo
    type: linux-bridge
    state: up
    mtu: 1400
    bridge:
      options:
        stp:
          enabled: false


Actual results:
vethfoo1 is no longer a port of brfoo

Expected results:
the vethfoo1 has to be still attached to the port.


Additional info:

This happend also changing STP or vlan-filtering, possible with more attributes

It also reproduces with nmcli

[root@worker1 ~]# bridge link show |grep br100
395: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 master br100 state forwarding priority 32 cost 100
404: vethf11c72f0@enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br100 state forwarding priority 32 cost 2
[root@worker1 ~]# nmcli con show br100|grep -i mtu
[root@worker1 ~]# nmcli c modify br100 ethernet.mtu 1600
[root@worker1 ~]# nmcli con up br100
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/122)
[root@worker1 ~]# bridge link show |grep br100
395: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 master br100 state forwarding priority 32 cost 100

Comment 1 Gris Ge 2022-06-22 06:20:50 UTC
Changing to NetworkManager after discussion with NM team.

To be more precise, we are requesting Reapply support of these options:

 * bridge.ageing-time
 * bridge.forward-delay
 * bridge.group-address
 * bridge.group-forward-mask
 * bridge.hello-time
 * bridge.max-age
 * bridge.multicast-hash-max
 * bridge.multicast-last-member-count
 * bridge.multicast-last-member-interval
 * bridge.multicast-membership-interval
 * bridge.multicast-querier
 * bridge.multicast-querier-interval
 * bridge.multicast-query-interval
 * bridge.multicast-query-response-interval
 * bridge.multicast-query-use-ifaddr
 * bridge.multicast-router
 * bridge.multicast-snooping
 * bridge.multicast-startup-query-count
 * bridge.multicast-startup-query-interval
 * bridge.priority
 * bridge.stp
 * bridge.vlan-filtering
 * bridge.vlan-protocol
 * bridge.vlans

 * 802-3-ethernet.accept-all-mac-addresses
 * 802-3-ethernet.cloned-mac-address
 * 802-3-ethernet.mtu
 * 802-3-ethernet.port

 * ipv4.addresses
 * ipv4.dhcp-client-id
 * ipv4.dhcp-iaid
 * ipv4.dhcp-timeout
 * ipv4.dns
 * ipv4.dns-priority
 * ipv4.dns-search
 * ipv4.gateway
 * ipv4.ignore-auto-dns
 * ipv4.ignore-auto-routes
 * ipv4.may-fail
 * ipv4.method
 * ipv4.never-default
 * ipv4.route-table
 * ipv4.routes
 * ipv4.routing-rules

 * ipv6.addr-gen-mode
 * ipv6.addresses
 * ipv6.dhcp-duid
 * ipv6.dhcp-iaid
 * ipv6.dhcp-timeout
 * ipv6.dns
 * ipv6.dns-priority
 * ipv6.dns-search
 * ipv6.gateway
 * ipv6.ignore-auto-dns
 * ipv6.may-fail
 * ipv6.method
 * ipv6.never-default
 * ipv6.ra-timeout
 * ipv6.route-metric
 * ipv6.route-table
 * ipv6.routes
 * ipv6.routing-rules

Comment 2 Gris Ge 2022-06-22 06:21:44 UTC
Hi Quique Llorente,

Can this be done in RHEL 9 only or you need both for 8 and 9?

Comment 3 Quique Llorente 2022-06-22 07:35:03 UTC
(In reply to Gris Ge from comment #2)
> Hi Quique Llorente,
> 
> Can this be done in RHEL 9 only or you need both for 8 and 9?

@bnemec we need the backport to RHEL8 right for standalone operator right ?

Comment 4 Ben Nemec 2022-06-22 21:42:31 UTC
Yes. Ideally we would backport to 4.10, which is RHEL 8-based (as is 4.11, I believe 4.12 is when we're planning to move to RHEL 9).

Comment 5 Thomas Haller 2022-06-23 12:42:39 UTC
(In reply to Gris Ge from comment #1)
> Changing to NetworkManager after discussion with NM team.
> 
> To be more precise, we are requesting Reapply support of these options:
> 
>  * bridge.ageing-time
>  * bridge.forward-delay
>  * bridge.group-address
>  * bridge.group-forward-mask
>  * bridge.hello-time
>  * bridge.max-age
>  * bridge.multicast-hash-max
>  * bridge.multicast-last-member-count
>  * bridge.multicast-last-member-interval
>  * bridge.multicast-membership-interval
>  * bridge.multicast-querier
>  * bridge.multicast-querier-interval
>  * bridge.multicast-query-interval
>  * bridge.multicast-query-response-interval
>  * bridge.multicast-query-use-ifaddr
>  * bridge.multicast-router
>  * bridge.multicast-snooping
>  * bridge.multicast-startup-query-count
>  * bridge.multicast-startup-query-interval
>  * bridge.priority
>  * bridge.stp
>  * bridge.vlan-filtering
>  * bridge.vlan-protocol
>  * bridge.vlans

It's probably sensible and doable to support reapply of all bridge properties.
In particular, if we have Fernando's rework of setting bridge options via netlink.

>  * 802-3-ethernet.accept-all-mac-addresses
>  * 802-3-ethernet.cloned-mac-address

cloned-mac-address seems not that it should be reapplied. The MAC address is a very fundamental property. Changing it without doing a full re-activation seems difficult -- to a point where we might want to require a full activation. For example, the IPv4 DHCP client-id defaults to the MAC address, if the MAC address changes we will probably get a different IP address too.

Now, during reapply we already restart DHCP, so it's not really a problem that the client-id and the IP address is going to change (in particular, as we already support reapply for enabling/disabling DHCP and changing the client-id). But still...

Is this necessary? 

>  * 802-3-ethernet.mtu
>  * 802-3-ethernet.port

I don't think that "port" is even functional (or useful).

> 
>  * ipv4.addresses
>  * ipv4.dhcp-client-id
>  * ipv4.dhcp-iaid
>  * ipv4.dhcp-timeout
>  * ipv4.dns
>  * ipv4.dns-priority
>  * ipv4.dns-search
>  * ipv4.gateway
>  * ipv4.ignore-auto-dns
>  * ipv4.ignore-auto-routes
>  * ipv4.may-fail
>  * ipv4.method
>  * ipv4.never-default
>  * ipv4.route-table
>  * ipv4.routes
>  * ipv4.routing-rules
> 
>  * ipv6.addr-gen-mode
>  * ipv6.addresses
>  * ipv6.dhcp-duid
>  * ipv6.dhcp-iaid
>  * ipv6.dhcp-timeout
>  * ipv6.dns
>  * ipv6.dns-priority
>  * ipv6.dns-search
>  * ipv6.gateway
>  * ipv6.ignore-auto-dns
>  * ipv6.may-fail
>  * ipv6.method
>  * ipv6.never-default
>  * ipv6.ra-timeout
>  * ipv6.route-metric
>  * ipv6.route-table
>  * ipv6.routes
>  * ipv6.routing-rules

ACK to the rest. Many of those should already work (like all the ipv4/ipv6 settings and the MTU).

Comment 6 Gris Ge 2022-06-27 13:44:59 UTC
These can be bemoved from reapply allowlist:

802-3-ethernet.cloned-mac-address
802-3-ethernet.ports

Thanks.

Comment 7 Gris Ge 2022-07-11 11:51:09 UTC
Bug https://bugzilla.redhat.com/show_bug.cgi?id=2105976 is created to tracking the reapply effort of `802-3-ethernet.cloned-mac-address`.

For `802-3-ethernet.port`, we do not have use case yet.

Comment 10 Gris Ge 2022-07-26 03:02:41 UTC
Hi Lubomir,

The `bridge.vlan-filtering` reapply is not working on NetworkManager-1.39.10-30745.copr.8e8fed433f.el9.x86_64. This is the output:


[fge@c9s ~]$ sudo nmcli c add type bridge ifname br0 bridge.vlan-filtering on connection.id br0
Connection 'br0' (54535fe1-cd78-446e-bd29-a673fa0fa99e) successfully added.
[fge@c9s ~]$ cat /sys/class/net/br0/bridge/vlan_
vlan_filtering       vlan_protocol        vlan_stats_enabled   vlan_stats_per_port
[fge@c9s ~]$ cat /sys/class/net/br0/bridge/vlan_filtering
1
[fge@c9s ~]$ sudo nmcli c modify br0 bridge.vlan-filtering off
[fge@c9s ~]$ sudo nmcli d reapply br0
Connection successfully reapplied to device 'br0'.
[fge@c9s ~]$ cat /sys/class/net/br0/bridge/vlan_filtering
1
[fge@c9s ~]$ sudo nmcli c up br0
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/20)
[fge@c9s ~]$ cat /sys/class/net/br0/bridge/vlan_filtering
0

Comment 17 errata-xmlrpc 2022-11-08 10:10:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (NetworkManager bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:7680