Bug 2065049

Description of problem:
If a bond connection changes the value of connection.autoconnect-slaves from -1 (undefined) to 1 (yes), NetworkManager cannot reapply the change to the device and brings the bond interface down an up, causing a loss of connectivity.

This has been observed in a bond of a RHVH hypervisor originally installed with version 4.4.5 (RHEL 8.3):

1. The bond was created at installation time by Anaconda with autoconnect-slaves=-1
2. Further modifications of the connection by RHV using nmstate-0.3.4-27.el8_3.noarch didn't modify the bond, which remained with autoconnect-slaves=-1
3. After upgrading the host to the latest version 4.4.10, nmstate was updated to nmstate-1.1.0-5.el8_5.noarch. In this version, adding a new VLAN to to the bond, caused it to be reconfigured with autoconnect-slaves=1
4. When nmstate applies the new settings, NetworkManager brings down the bond, causing a network outage in the host and in all the VMs running in VLANs dependant of the bond device.


Version-Release number of selected component (if applicable):
NetworkManager-1.32.10-4.el8.x86_64
nmstate-1.1.0-5.el8_5.noarch
kernel-4.18.0-348.12.2.el8_5.x86_64

How reproducible:
Always 

Steps to Reproduce:
1. In a RHEL 8.5 machine, create a bond with connection.autoconnect-slaves unset:

# nmcli con show bond0 |grep connection.autoconnect-slaves
connection.autoconnect-slaves:          -1 (default)

2. Set the option connection.autoconnect-slaves=yes

# nmcli con modify bond0 connection.autoconnect-slaves yes

3. Reapply the settings to the device:

# nmcli device reapply bond0
Error: Reapplying connection to device 'bond0' (/org/freedesktop/NetworkManager/Devices/7) failed: Can't reapply changes to 'connection.autoconnect-slaves' setting

Actual results:
The bond interface and all dependendant interfaces are brought down:

Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9027] audit: op="connection-update" uuid="763cb068-b9bb-4ae5-b483-a5b5befb91de" name="bond0" args="connection.master,connection.autoconnect-slaves" pid=2022 uid=0 result="success"
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9039] audit: op="connection-update" uuid="206a3829-9919-4958-b9c4-d77e5d46c2cd" name="ovirtmgmt" args="ipv4.route-table,ipv6.route-table" pid=2022 uid=0 result="success"
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9080] audit: op="device-reapply" interface="ovirtmgmt" ifindex=8 args="ipv4.route-table,ipv6.route-table" pid=2022 uid=0 result="success"
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9110] audit: op="device-reapply" interface="bond0" ifindex=10 args="connection.master,connection.autoconnect-slaves" pid=2022 uid=0 result="fail" reason="Can't reapply changes to 'connection.master' setting"
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9124] device (bond0): state change: activated -> deactivating (reason 'new-activation', sys-iface-state: 'managed')
Mar 16 09:13:12 rhvh445-01 kernel: ovirtmgmt: port 1(bond0) entered disabled state
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9212] device (ovirtmgmt): detached bridge port bond0
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9212] device (bond0): released from master device ovirtmgmt
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9218] device (bond0): disconnecting for new activation request.
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9218] audit: op="connection-activate" uuid="763cb068-b9bb-4ae5-b483-a5b5befb91de" name="bond0" pid=2022 uid=0 result="success"
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9219] device (enp1s0): state change: activated -> deactivating (reason 'new-activation', sys-iface-state: 'managed')
Mar 16 09:13:12 rhvh445-01 kernel: bond0: (slave enp1s0): Releasing backup interface
Mar 16 09:13:12 rhvh445-01 kernel: bond0: (slave enp1s0): the permanent HWaddr of slave - 52:54:00:18:95:ce - is still in use by bond - set the HWaddr of slave to a different address to avoid conflicts
Mar 16 09:13:12 rhvh445-01 kernel: device enp1s0 left promiscuous mode
Mar 16 09:13:12 rhvh445-01 kernel: bond0: (slave enp8s0): making interface the new active one
Mar 16 09:13:12 rhvh445-01 kernel: device enp8s0 entered promiscuous mode
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9382] device (bond0): released bond slave enp1s0
Mar 16 09:13:12 rhvh445-01 kernel: 8021q: adding VLAN 0 to HW filter on device enp1s0
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9416] device (enp1s0): released from master device bond0
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9431] device (enp1s0): disconnecting for new activation request.
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9432] device (enp8s0): state change: activated -> deactivating (reason 'new-activation', sys-iface-state: 'managed')
Mar 16 09:13:12 rhvh445-01 kernel: bond0: (slave enp8s0): Releasing backup interface
Mar 16 09:13:12 rhvh445-01 kernel: device enp8s0 left promiscuous mode
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9526] device (bond0): released bond slave enp8s0
Mar 16 09:13:12 rhvh445-01 kernel: test1: port 1(bond0.20) entered disabled state
Mar 16 09:13:12 rhvh445-01 kernel: 8021q: adding VLAN 0 to HW filter on device enp8s0
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9559] device (bond0.20): set-hw-addr: set MAC address to 52:0F:96:CE:BF:D3 (vlan-parent)
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9564] device (bond0.20): set-hw-addr: set MAC address to 52:54:00:18:95:CE (vlan-parent)
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9564] device (bond0): set-hw-addr: set MAC address to 52:54:00:18:95:CE (restore)
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9576] device (enp8s0): released from master device bond0
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9582] device (enp8s0): disconnecting for new activation request.
Mar 16 09:13:12 rhvh445-01 NetworkManager[1490]: <info>  [1647421992.9598] device (bond0): state change: deactivating -> disconnected (reason 'new-activation', sys-iface-state: 'managed')


Expected results:
The interface is not brought down. IMO autoconnect-slaves is not an actual setting that needs to be applied to the interface, but more of a management setting in NetworkManager's internal state for the connection.

Additional info: