Description of problem: I'm having some problems with a configuration where an active-backup bond has an 802.3ad bond as a slave. Both bonds often end up with a random MAC address after booting due to what seems to be a race condition. See https://bugzilla.redhat.com/show_bug.cgi?id=2188100 . So I thought I could work around the race condition by bringing up the 802.3ad bond first, and then waiting for it to stabilize before bringing up the active-backup bond layered on top. So I edited /etc/sysconfig/network-scripts/ifcfg-bond1 to set ONBOOT=no for bond1 (the active-backup bond), as well as ifcfg-bond1.3 However, after rebooting, I see that NetworkManager started bond1 despite being told not to do so. Version-Release number of selected component (if applicable): NetworkManager-1.43.4-1.el9.x86_64 How reproducible: always Steps to Reproduce: 1. create 802.3ad bond0 and active-backup bond1 with bond1 ONBOOT=no 2. reboot 3. Actual results: Notice that NetworkManager started bond1 Expected results: It should not start bond1. Additional info: I noticed some similarly strange issues in the way NetworkManager is linking these 2 bonds together. When I configure ONBOOT=yes for both bonds, I usually get a random MAC address. I thought I could run 'ifdown bond1; ifdown bond0; ifup bond0' and that this would give bond0 a chance to reassign the MAC address properly from one of its slave ethernet devices. But this behaves strangely. When I run 'ifdown bond1', it also removes the bond0 interface. I don't know why it does this. And if I then run 'ifup bond0', it automatically activates bond1 as well. Why is that happening? I don't understand the linkage. I see a "connection.autoconnect-slaves" setting, which looks maybe relevant, but it's set to -1, which I think is the default, which should map to 0, shouln't it? What's telling it to auto-disconnect the slaves when I ifdown bond1? And why would it auto-connect the master (implicitly ifup bond1) when I run ifup bond0? This seems flaky to me. Would I get better results if I migrated to the /etc/NetworkManager/system-connections format? I can't see why it should matter.
What gives `nmcli connection` and `nmcli device`? Please provide a logfile of the boot. Get it via `journalctl -u NetworkManager`. Thank you.
Created attachment 1959545 [details] NetworkManager journal messages sh-5.1# nmcli connection NAME UUID TYPE DEVICE Vlan bond1.3 2682acff-f7af-8f05-0172-f959526caa05 vlan bond1.3 lo baf144b5-9b68-4d65-a64c-4ccc4a11c81e loopback lo Bond bond1 92306dc1-4142-23de-097b-b1464cfab5ee bond bond1 Bond bond0 ad33d8b0-1f7b-cab9-9447-ba07f855b143 bond bond0 System lan0 1969c5f0-fbcf-2bd8-5a7e-7a8a67908f8f ethernet lan0 System lan2 38ac5fa0-df4c-251b-7afb-02fcb93ed035 ethernet lan2 System lan3 93ad060f-9feb-77e2-8938-1b17f04e706e ethernet lan3 sh-5.1# nmcli device DEVICE TYPE STATE CONNECTION bond1.3 vlan connected Vlan bond1.3 lo loopback connected lo bond1 bond connected Bond bond1 bond0 bond connected Bond bond0 lan0 ethernet connected System lan0 lan2 ethernet connected System lan2 lan3 ethernet connected System lan3 lan1 ethernet unmanaged -- lan4 ethernet unmanaged -- lan5 ethernet unmanaged -- sh-5.1# journalctl -b -u NetworkManager > /tmp/nmboot.txt I have attached nmboot.txt In case it matters, I ran 'ifup bond1.3' after booting.
To be clear, in case it's confusing from the boot messages, I created a local systemd service that runs 'ifup bond1.3' immediately after NetworkManager-wait-online.service finishes and before network-online.target. NetworkManager is (correctly) not starting that one because I have ONBOOT=no. But it is starting bond1 despite ONBOOT=no: sh-5.1$ cd /etc/sysconfig/network-scripts/ sh-5.1$ egrep 'ONBOOT|NM_CONTROL' * ifcfg-bond0:ONBOOT=yes ifcfg-bond0:NM_CONTROLLED=yes ifcfg-bond1:ONBOOT=no ifcfg-bond1:NM_CONTROLLED=yes ifcfg-bond1.3:ONBOOT=no ifcfg-bond1.3:NM_CONTROLLED=yes ifcfg-lan0:ONBOOT=yes ifcfg-lan0:NM_CONTROLLED=yes ifcfg-lan1:ONBOOT=no ifcfg-lan1:NM_CONTROLLED=no ifcfg-lan2:ONBOOT=yes ifcfg-lan2:NM_CONTROLLED=yes ifcfg-lan3:ONBOOT=yes ifcfg-lan3:NM_CONTROLLED=yes ifcfg-lan4:ONBOOT=no ifcfg-lan4:NM_CONTROLLED=no ifcfg-lan5:ONBOOT=no ifcfg-lan5:NM_CONTROLLED=no And here's my kludge to bring up bond1.3: [Unit] Description=Bring up stacked bonds after lower bonds have stabilized After=NetworkManager-wait-online.service network.target Before=network-online.target [Service] Type=oneshot ExecStart=/usr/sbin/ifup bond1.3 [Install] WantedBy=multi-user.target Somehow, this approach is fixing the problem whereby I was getting a random MAC address on the bonds. Perhaps setting ONBOOT=no is at least delaying the startup of bond1 a bit. It's apparently enough to give bond0 time to get a valid MAC address. Or so it seems from 5 or so reboots. But I'd prefer if NetworkManager didn't start bond1 so I could have more control over this situation and be confident that I can defeat the race condition.
> But it is starting bond1 despite ONBOOT=no: > > [1682283629.2125] policy: auto-activating connection 'System lan0' (1969c5f0-fbcf-2bd8-5a7e-7a8a67908f8f) > ... > [1682283629.2789] device (bond1): attached bond port lan0 bond1 is activated already before, because you configured a port profile "System lan0" which is set to autoconnect. (Auto-)activating a port device/profile implies to bring up the controller bond interface. If you don't want those profiles to auto-activate, then don't configure them to autoconnect. On top of that, bond0 device has master=bond1: > <info> [1682283629.4195] device (bond1): attached bond port bond0 So for that reason also, activating bond0 will cause bond1 to be up. Adding a systemd service for calling ifup seems not like any solution. If your problem is that an interface doesn't get the right MAC address, then let's not focus on how you'd like to manually connect interfaces (or how an interface is unexecptedly autoconnected), but rather how you can configure your system so that it works. Configure suitable profiles. See all profiles with `nmcli connection`. See the settings of a profile with `nmcli connection show "$PROFILE_NAME"` (or `nmcli -o connection show "$PROFILE_NAME"). Modify the profile with `nmcli connection modify "$PROFILE_NAME" ...`. Worst case, if you need a specific MAC address, you could configure it with `nmcli connection modify "$PROFILE_NAME" ethernet.cloned-mac-address "$MAC"`. Otherwise, the MAC address is chosen by kernel, and it depends on the bonding mode ("active-backup"). Consider setting the "primary" bond option. nmcli connection modify 'Bond bond1' +bond.options 'primary=lan0' If you don't get it to work, then share - nmcli -f all connection - nmcli device - ip -d link - grep -R ^ /etc/NetworkManager/system-connections /etc/sysconfig/network-scripts - collect a complete `level=TRACE` log from a current boot. See DEBUGGING in `man NetworkManager`. You'll find the log in `journalctl -b0` output. BEWARE the hint in the manual page about sharing private data, before sharing.
OK, so you are saying that bringing up a slave interface will always implicitly bring up the master interface. I guess that maybe makes sense, but it doesn't seem inevitable. The root cause problem here is that I want an active-backup bond in bond1 with one of its slaves bond0 to be an 802.3ad bond, and NetworkManager has a race condition when bringing up these interfaces. I'm not certain, but I believe what's going on is that the 802.3ad bond in bond0 takes a while to come up because it needs to exchange LACP packets with the switch. Until its first slave becomes active, I guess bond0 has a random MAC address assigned. But bond1 is coming up before bond0 is stabilized, and bond1 steals the MAC address from bond0, which is random at that moment. I'm not aware of any options I could set anywhere to fix this race condition. The only solution I can see is to delay starting bond1 until bond0 stabilizes, i.e. finishes negotiating LACP with the switch on at least one link. And in fact, setting bond1 ONBOOT=no does seem to delay starting bond1 just enough to solve my problem, although that may just be luck from a small sample size. What possible setting do you have in mind that could convince NetworkManager not to start bond1 until after bond0 finishes negotiating LACP and has a reasonable MAC address? Thanks, Andy P.S. And yes -- I agree that my solution to add a systemd service to call ifup is a hack. But I need a hack that works, because I don't see how to get NetworkManager to do the right thing. I think that NetworkManager is fundamentally too stupid to grok this configuration correctly. But I'm certainly no expert in NetworkManager, and I've love to be convinced that I'm wrong. What hints could I gives to NetworkManager to tell it to delay starting bond1 until bond0 has stabilized with a good MAC address? Alternatively, if I could convince bond1 to steal the MAC address from lan0 instead of bond0, that could fix the issue. But I don't see how to do that. In practice, it seems to randomly select a MAC address from either lan0 or bond0, and if bond0 hasn't stabilized in time, its MAC address will be random. P.P.S. I am setting the primary slave. I don't know why that would matter. The primary slave is bond0, since that has the bandwidth. Here are my config files: sh-5.1$ cd /etc/sysconfig/network-scripts/ sh-5.1$ head -20 ifcfg-* ==> ifcfg-bond0 <== DEVICE=bond0 TYPE=Bond BOOTPROTO=none ONBOOT=yes HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=yes BONDING_MASTER=yes BONDING_OPTS="miimon=100 ad_select=bandwidth mode=802.3ad xmit_hash_policy=layer2+3 arp_interval=0 lacp_rate=fast" MASTER=bond1 SLAVE=yes ==> ifcfg-bond1 <== DEVICE=bond1 TYPE=Bond BOOTPROTO=static ONBOOT=no HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=yes IPADDR=192.168.30.27 NETMASK=255.255.254.0 BONDING_MASTER=yes BONDING_OPTS="miimon=0 mode=active-backup arp_all_targets=any primary=bond0 arp_ip_target=192.168.30.13,192.168.30.12,192.168.30.7,192.168.30.5 arp_interval=1000 fail_over_mac=follow arp_validate=none primary_reselect=always" ==> ifcfg-bond1.3 <== DEVICE=bond1.3 BOOTPROTO=static ONBOOT=no HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=yes VLAN=yes IPADDR=192.168.33.27 NETMASK=255.255.255.0 ==> ifcfg-lan0 <== DEVICE=lan0 TYPE=Ethernet BOOTPROTO=none ONBOOT=yes HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=yes MASTER=bond1 SLAVE=yes ==> ifcfg-lan1 <== DEVICE=lan1 ONBOOT=no HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=no ==> ifcfg-lan2 <== DEVICE=lan2 TYPE=Ethernet BOOTPROTO=none ONBOOT=yes HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=yes MASTER=bond0 SLAVE=yes ==> ifcfg-lan3 <== DEVICE=lan3 TYPE=Ethernet BOOTPROTO=none ONBOOT=yes HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=yes MASTER=bond0 SLAVE=yes ==> ifcfg-lan4 <== DEVICE=lan4 ONBOOT=no HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=no ==> ifcfg-lan5 <== DEVICE=lan5 ONBOOT=no HOTPLUG=no NOZEROCONF=yes NM_CONTROLLED=no
Anyway, by setting ONBOOT=no for bond1 and bond1.3, it seems somehow to delay bringing up bond1 just enough to fix my issue. And then my kludge systemd service starts bond1.3 to get things to the proper state before network-online.target. But here's the various other debugging info you requested: sh-5.1$ nmcli -f all connection | cat - NAME UUID TYPE TIMESTAMP TIMESTAMP-REAL AUTOCONNECT AUTOCONNECT-PRIORITY READONLY DBUS-PATH ACTIVE DEVICE STATE ACTIVE-PATH SLAVE FILENAME Vlan bond1.3 2682acff-f7af-8f05-0172-f959526caa05 vlan 1682470608 Tue Apr 25 20:56:48 2023 yes 0 no /org/freedesktop/NetworkManager/Settings/6 yes bond1.3 activated /org/freedesktop/NetworkManager/ActiveConnection/7 -- /etc/sysconfig/network-scripts/ifcfg-bond1.3 lo 1bc71bbb-7bfc-458b-8987-dc989623ed73 loopback 1682470607 Tue Apr 25 20:56:47 2023 no 0 no /org/freedesktop/NetworkManager/Settings/7 yes lo activated /org/freedesktop/NetworkManager/ActiveConnection/1 -- /run/NetworkManager/system-connections/lo.nmconnection Bond bond1 92306dc1-4142-23de-097b-b1464cfab5ee bond 1682470608 Tue Apr 25 20:56:48 2023 yes 0 no /org/freedesktop/NetworkManager/Settings/5 yes bond1 activated /org/freedesktop/NetworkManager/ActiveConnection/5 -- /etc/sysconfig/network-scripts/ifcfg-bond1 Bond bond0 ad33d8b0-1f7b-cab9-9447-ba07f855b143 bond 1682470607 Tue Apr 25 20:56:47 2023 yes 0 no /org/freedesktop/NetworkManager/Settings/1 yes bond0 activated /org/freedesktop/NetworkManager/ActiveConnection/6 bond /etc/sysconfig/network-scripts/ifcfg-bond0 System lan0 1969c5f0-fbcf-2bd8-5a7e-7a8a67908f8f ethernet 1682470607 Tue Apr 25 20:56:47 2023 yes 0 no /org/freedesktop/NetworkManager/Settings/4 yes lan0 activated /org/freedesktop/NetworkManager/ActiveConnection/2 bond /etc/sysconfig/network-scripts/ifcfg-lan0 System lan2 38ac5fa0-df4c-251b-7afb-02fcb93ed035 ethernet 1682470607 Tue Apr 25 20:56:47 2023 yes 0 no /org/freedesktop/NetworkManager/Settings/2 yes lan2 activated /org/freedesktop/NetworkManager/ActiveConnection/3 bond /etc/sysconfig/network-scripts/ifcfg-lan2 System lan3 93ad060f-9feb-77e2-8938-1b17f04e706e ethernet 1682470607 Tue Apr 25 20:56:47 2023 yes 0 no /org/freedesktop/NetworkManager/Settings/3 yes lan3 activated /org/freedesktop/NetworkManager/ActiveConnection/4 bond /etc/sysconfig/network-scripts/ifcfg-lan3 sh-5.1$ nmcli device DEVICE TYPE STATE CONNECTION bond1.3 vlan connected Vlan bond1.3 lo loopback connected (externally) lo bond1 bond connected Bond bond1 bond0 bond connected Bond bond0 lan0 ethernet connected System lan0 lan2 ethernet connected System lan2 lan3 ethernet connected System lan3 lan1 ethernet unmanaged -- lan4 ethernet unmanaged -- lan5 ethernet unmanaged -- sh-5.1$ ip -d link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 minmtu 0 maxmtu 0 addrgenmode none numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 524280 tso_max_segs 65535 gro_max_size 65536 2: lan0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond1 state UP mode DEFAULT group default qlen 1000 link/ether 40:a6:b7:b0:b7:c0 brd ff:ff:ff:ff:ff:ff permaddr a0:36:bc:c8:07:a9 promiscuity 0 minmtu 68 maxmtu 9216 bond_slave state BACKUP mii_status UP link_failure_count 0 perm_hwaddr a0:36:bc:c8:07:a9 queue_id 0 addrgenmode none numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 parentbus pci parentdev 0000:05:00.0 altname enp5s0 3: lan1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether a0:36:bc:c8:07:aa brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 9216 addrgenmode eui64 numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 parentbus pci parentdev 0000:06:00.0 altname enp6s0 4: lan2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether a6:c7:fb:56:3d:3d brd ff:ff:ff:ff:ff:ff permaddr 40:a6:b7:b0:b7:c0 promiscuity 0 minmtu 68 maxmtu 9702 bond_slave state ACTIVE mii_status UP link_failure_count 1 perm_hwaddr 40:a6:b7:b0:b7:c0 queue_id 0 ad_aggregator_id 2 ad_actor_oper_port_state 63 ad_actor_oper_port_state_str <active,short_timeout,aggregating,in_sync,collecting,distributing> ad_partner_oper_port_state 61 ad_partner_oper_port_state_str <active,aggregating,in_sync,collecting,distributing> addrgenmode none numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 portid 40a6b7b0b7c0 parentbus pci parentdev 0000:01:00.0 5: lan3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether a6:c7:fb:56:3d:3d brd ff:ff:ff:ff:ff:ff permaddr 40:a6:b7:b0:b7:c1 promiscuity 0 minmtu 68 maxmtu 9702 bond_slave state ACTIVE mii_status UP link_failure_count 1 perm_hwaddr 40:a6:b7:b0:b7:c1 queue_id 0 ad_aggregator_id 2 ad_actor_oper_port_state 63 ad_actor_oper_port_state_str <active,short_timeout,aggregating,in_sync,collecting,distributing> ad_partner_oper_port_state 61 ad_partner_oper_port_state_str <active,aggregating,in_sync,collecting,distributing> addrgenmode none numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 portid 40a6b7b0b7c1 parentbus pci parentdev 0000:01:00.1 6: lan4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 40:a6:b7:b0:b7:c2 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 9702 addrgenmode eui64 numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 portid 40a6b7b0b7c2 parentbus pci parentdev 0000:01:00.2 7: lan5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 40:a6:b7:b0:b7:c3 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 9702 addrgenmode eui64 numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 portid 40a6b7b0b7c3 parentbus pci parentdev 0000:01:00.3 8: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether a6:c7:fb:56:3d:3d brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65535 bond mode active-backup active_slave bond0 miimon 0 updelay 0 downdelay 0 peer_notify_delay 0 use_carrier 1 arp_interval 1000 arp_missed_max 2 arp_ip_target 192.168.30.13,192.168.30.12,192.168.30.7,192.168.30.5 arp_validate none arp_all_targets any primary bond0 primary_reselect always fail_over_mac follow xmit_hash_policy layer2 resend_igmp 1 num_grat_arp 1 all_slaves_active 0 min_links 0 lp_interval 1 packets_per_slave 1 lacp_active on lacp_rate slow ad_select stable tlb_dynamic_lb 1 addrgenmode eui64 numtxqueues 16 numrxqueues 16 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 9: bond1.3@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether a6:c7:fb:56:3d:3d brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 0 maxmtu 65535 vlan protocol 802.1Q id 3 <REORDER_HDR> addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 10: bond0: <BROADCAST,MULTICAST,MASTER,SLAVE,UP,LOWER_UP> mtu 1500 qdisc noqueue master bond1 state UP mode DEFAULT group default qlen 1000 link/ether a6:c7:fb:56:3d:3d brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65535 bond mode 802.3ad miimon 100 updelay 0 downdelay 0 peer_notify_delay 0 use_carrier 1 arp_interval 0 arp_missed_max 2 arp_validate none arp_all_targets any primary_reselect always fail_over_mac none xmit_hash_policy layer2+3 resend_igmp 1 num_grat_arp 1 all_slaves_active 0 min_links 0 lp_interval 1 packets_per_slave 1 lacp_active on lacp_rate fast ad_select bandwidth ad_aggregator 2 ad_num_ports 2 ad_actor_key 15 ad_partner_key 2 ad_partner_mac 10:c5:95:02:10:00 tlb_dynamic_lb 1 bond_slave state ACTIVE mii_status UP link_failure_count 0 perm_hwaddr a6:c7:fb:56:3d:3d queue_id 0 addrgenmode none numtxqueues 16 numrxqueues 16 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 sh-5.1$ grep -R ^ /etc/NetworkManager/system-connections /etc/sysconfig/network-scripts /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:NetworkManager stores new network profiles in keyfile format in the /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:/etc/NetworkManager/system-connections/ directory. /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:Previously, NetworkManager stored network profiles in ifcfg format /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:in this directory (/etc/sysconfig/network-scripts/). However, the ifcfg /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:format is deprecated. By default, NetworkManager no longer creates /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:new profiles in this format. /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:Connection profiles in keyfile format have many benefits. For example, /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:this format is INI file-based and can easily be parsed and generated. /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:Each section in NetworkManager keyfiles corresponds to a NetworkManager /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:setting name as described in the nm-settings(5) and nm-settings-keyfile(5) /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:man pages. Each key-value-pair in a section is one of the properties /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:listed in the settings specification of the man page. /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:If you still use network profiles in ifcfg format, consider migrating /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:them to keyfile format. To migrate all profiles at once, enter: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:# nmcli connection migrate /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:This command migrates all profiles from ifcfg format to keyfile /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:format and stores them in /etc/NetworkManager/system-connections/. /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:Alternatively, to migrate only a specific profile, enter: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:# nmcli connection migrate <profile_name|UUID|D-Bus_path> /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:For further details, see: /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:* nm-settings-keyfile(5) /etc/sysconfig/network-scripts/readme-ifcfg-rh.txt:* nmcli(1) /etc/sysconfig/network-scripts/route-bond1.3:default via 192.168.33.254 dev bond1.3 metric 4 /etc/sysconfig/network-scripts/ifcfg-bond0:DEVICE=bond0 /etc/sysconfig/network-scripts/ifcfg-bond0:TYPE=Bond /etc/sysconfig/network-scripts/ifcfg-bond0:BOOTPROTO=none /etc/sysconfig/network-scripts/ifcfg-bond0:ONBOOT=yes /etc/sysconfig/network-scripts/ifcfg-bond0:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-bond0:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-bond0:NM_CONTROLLED=yes /etc/sysconfig/network-scripts/ifcfg-bond0:BONDING_MASTER=yes /etc/sysconfig/network-scripts/ifcfg-bond0:BONDING_OPTS="miimon=100 ad_select=bandwidth mode=802.3ad xmit_hash_policy=layer2+3 arp_interval=0 lacp_rate=fast" /etc/sysconfig/network-scripts/ifcfg-bond0:MASTER=bond1 /etc/sysconfig/network-scripts/ifcfg-bond0:SLAVE=yes /etc/sysconfig/network-scripts/ifcfg-lan2:DEVICE=lan2 /etc/sysconfig/network-scripts/ifcfg-lan2:TYPE=Ethernet /etc/sysconfig/network-scripts/ifcfg-lan2:BOOTPROTO=none /etc/sysconfig/network-scripts/ifcfg-lan2:ONBOOT=yes /etc/sysconfig/network-scripts/ifcfg-lan2:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-lan2:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-lan2:NM_CONTROLLED=yes /etc/sysconfig/network-scripts/ifcfg-lan2:MASTER=bond0 /etc/sysconfig/network-scripts/ifcfg-lan2:SLAVE=yes /etc/sysconfig/network-scripts/ifcfg-lan3:DEVICE=lan3 /etc/sysconfig/network-scripts/ifcfg-lan3:TYPE=Ethernet /etc/sysconfig/network-scripts/ifcfg-lan3:BOOTPROTO=none /etc/sysconfig/network-scripts/ifcfg-lan3:ONBOOT=yes /etc/sysconfig/network-scripts/ifcfg-lan3:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-lan3:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-lan3:NM_CONTROLLED=yes /etc/sysconfig/network-scripts/ifcfg-lan3:MASTER=bond0 /etc/sysconfig/network-scripts/ifcfg-lan3:SLAVE=yes /etc/sysconfig/network-scripts/ifcfg-lan0:DEVICE=lan0 /etc/sysconfig/network-scripts/ifcfg-lan0:TYPE=Ethernet /etc/sysconfig/network-scripts/ifcfg-lan0:BOOTPROTO=none /etc/sysconfig/network-scripts/ifcfg-lan0:ONBOOT=yes /etc/sysconfig/network-scripts/ifcfg-lan0:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-lan0:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-lan0:NM_CONTROLLED=yes /etc/sysconfig/network-scripts/ifcfg-lan0:MASTER=bond1 /etc/sysconfig/network-scripts/ifcfg-lan0:SLAVE=yes /etc/sysconfig/network-scripts/ifcfg-lan1:DEVICE=lan1 /etc/sysconfig/network-scripts/ifcfg-lan1:ONBOOT=no /etc/sysconfig/network-scripts/ifcfg-lan1:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-lan1:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-lan1:NM_CONTROLLED=no /etc/sysconfig/network-scripts/ifcfg-lan4:DEVICE=lan4 /etc/sysconfig/network-scripts/ifcfg-lan4:ONBOOT=no /etc/sysconfig/network-scripts/ifcfg-lan4:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-lan4:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-lan4:NM_CONTROLLED=no /etc/sysconfig/network-scripts/ifcfg-lan5:DEVICE=lan5 /etc/sysconfig/network-scripts/ifcfg-lan5:ONBOOT=no /etc/sysconfig/network-scripts/ifcfg-lan5:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-lan5:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-lan5:NM_CONTROLLED=no /etc/sysconfig/network-scripts/ifcfg-bond1:DEVICE=bond1 /etc/sysconfig/network-scripts/ifcfg-bond1:TYPE=Bond /etc/sysconfig/network-scripts/ifcfg-bond1:BOOTPROTO=static /etc/sysconfig/network-scripts/ifcfg-bond1:ONBOOT=yes /etc/sysconfig/network-scripts/ifcfg-bond1:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-bond1:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-bond1:NM_CONTROLLED=yes /etc/sysconfig/network-scripts/ifcfg-bond1:IPADDR=192.168.30.27 /etc/sysconfig/network-scripts/ifcfg-bond1:NETMASK=255.255.254.0 /etc/sysconfig/network-scripts/ifcfg-bond1:BONDING_MASTER=yes /etc/sysconfig/network-scripts/ifcfg-bond1:BONDING_OPTS="miimon=0 mode=active-backup arp_all_targets=any primary=bond0 arp_ip_target=192.168.30.13,192.168.30.12,192.168.30.7,192.168.30.5 arp_interval=1000 fail_over_mac=follow arp_validate=none primary_reselect=always" /etc/sysconfig/network-scripts/ifcfg-bond1.3:DEVICE=bond1.3 /etc/sysconfig/network-scripts/ifcfg-bond1.3:BOOTPROTO=static /etc/sysconfig/network-scripts/ifcfg-bond1.3:ONBOOT=yes /etc/sysconfig/network-scripts/ifcfg-bond1.3:HOTPLUG=no /etc/sysconfig/network-scripts/ifcfg-bond1.3:NOZEROCONF=yes /etc/sysconfig/network-scripts/ifcfg-bond1.3:NM_CONTROLLED=yes /etc/sysconfig/network-scripts/ifcfg-bond1.3:VLAN=yes /etc/sysconfig/network-scripts/ifcfg-bond1.3:IPADDR=192.168.33.27 /etc/sysconfig/network-scripts/ifcfg-bond1.3:NETMASK=255.255.255.0 And I'm uploading the level=TRACE log as an attachment called NetworkManager_trace.txt
Created attachment 1959940 [details] NetworkManager TRACE log This is the output from: journalctl -b0 -uNetworkManager I previously set level=TRACE in /etc/NetworkManager/NetworkManager.conf and in /etc/systemd/journald.conf: RateLimitIntervalSec=0 RateLimitBurst=0 For whatever it's worth, I believe that NetworkManager should not automatically start a master interface if the master has ONBOOT=no. I don't see why starting a slave means that one must implicitly start the master. But in any case, the ONBOOT=no hack seems to be working for me for now. Regards, Andy
Created attachment 1959941 [details] journalctl boot log In case you need all of the boot messages, here's the output from: journalctl -b0 Regards, Andy
Thank you for your detailed report and investigation. Considering the issue involves a complex bonding configuration with a race condition on booting and our current team's capacity, we've decided to close this issue as WONTFIX. It appears that your workaround is effective for the time being.