Bug 2188280 - MAC Address Policy none use wrong mac address
Summary: MAC Address Policy none use wrong mac address
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 38
Hardware: Unspecified
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2107754
TreeView+ depends on / blocked
 
Reported: 2023-04-20 10:53 UTC by bengt
Modified: 2023-04-20 13:11 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-04-20 13:11:38 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
requested log (211.92 KB, text/plain)
2023-04-20 11:26 UTC, bengt
no flags Details

Description bengt 2023-04-20 10:53:22 UTC
From changeset description (https://fedoraproject.org/wiki/Changes/MAC_Address_Policy_none)
"In the case of MACAddressPolicy=none for a bond (or bridge) the bond will get a MAC related to one of its physical NIC devices."

This is somewhat not what happpens with bridge interface. I you have only configured a bridge you are fine. Bridge get same mac as the "first" physical nic. But issue starts when you add virtual nic to that bridge, then machine use first virtual nic mac to request ip from dhcp.

A machine with following nic:
ip -br link
lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> 
enp3s0f0         UP             a8:60:b6:1d:86:17 <BROADCAST,MULTICAST,UP,LOWER_UP> 
bridge0          UP             a8:60:b6:1d:86:17 <BROADCAST,MULTICAST,UP,LOWER_UP> 
vnet0            UNKNOWN        fe:54:00:82:42:55 <BROADCAST,MULTICAST,UP,LOWER_UP>

bridge  link
2: enp3s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master bridge0 state forwarding priority 32 cost 100 
4: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master bridge0 state forwarding priority 32 cost 2

dhcpd log:
dhcpd[661]: DHCPOFFER on x.x.x.x to fe:54:00:82:42:55
dhcpd[661]: DHCPREQUEST for x.x.x.x from fe:54:00:82:42:55 (kvm-mini) via enabcm6e4ei0
dhcpd[661]: DHCPACK on x.x.x.x to fe:54:00:82:42:55

As you can see dhcp request is sendt for mac from "first" virtual nic connected to bridge.

What is strange is that bridge0 is given mac from physical nic. Can not say for sure if that happens later.

This behaviour makes it impossible to predict what mac is used to request ip from dhcp. Virtual nic can change, that is the nature of virtual nic.



Reproducible: Always

Steps to Reproduce:
1.Create a network bridge
2.Add a virtual nic to bridge
3.Check what mac is used for dhcp request
Actual Results:  
Unpredictable mac is used for dhcp request.

Expected Results:  
Predictable mac is used for dhcp request. This was one of main reasons for this change.

Comment 1 Zbigniew Jędrzejewski-Szmek 2023-04-20 11:16:16 UTC
Please attach a full log ('journalctl -b -o short-monotonic --no-hostname').

Comment 2 bengt 2023-04-20 11:18:56 UTC
from journal log:
kernel: bridge0: port 1(vnet0) entered blocking state
kernel: bridge0: port 1(vnet0) entered disabled state
kernel: bridge0: port 1(vnet0) entered blocking state
kernel: bridge0: port 1(vnet0) entered forwarding state
kernel: IPv6: ADDRCONF(NETDEV_CHANGE): bridge0: link becomes ready
NetworkManager[676]: <info>  [1681982573.9089] device (bridge0): carrier: link connected
NetworkManager[676]: <info>  [1681982573.9195] device (bridge0): bridge port vnet0 was attached
NetworkManager[676]: <info>  [1681982573.9197] dhcp4 (bridge0): activation: beginning transaction (timeout in 45 seconds)
kernel: bridge0: port 2(vnet1) entered blocking state
kernel: bridge0: port 2(vnet1) entered disabled state
kernel: bridge0: port 2(vnet1) entered blocking state
kernel: bridge0: port 2(vnet1) entered forwarding state
NetworkManager[676]: <info>  [1681982574.4039] device (bridge0): bridge port vnet1 was attached
kernel: bridge0: port 3(vnet2) entered blocking state
kernel: bridge0: port 3(vnet2) entered disabled state
kernel: bridge0: port 3(vnet2) entered blocking state
kernel: bridge0: port 3(vnet2) entered forwarding state
NetworkManager[676]: <info>  [1681982574.8577] device (bridge0): bridge port vnet2 was attached
kernel: bridge0: port 4(enp3s0f0) entered blocking state
kernel: bridge0: port 4(enp3s0f0) entered disabled state
kernel: bridge0: port 4(enp3s0f0) entered blocking state
kernel: bridge0: port 4(enp3s0f0) entered forwarding state
NetworkManager[676]: <info>  [1681982576.9356] device (bridge0): attached bridge port enp3s0f0
NetworkManager[676]: <info>  [1681982583.7999] dhcp4 (bridge0): state changed new lease, address=x.x.x.219
NetworkManager[676]: <info>  [1681982583.8004] policy: set 'bridge0' (bridge0) as default for IPv4 routing and DNS
systemd-resolved[612]: bridge0: Bus client set search domain list to: fredhs.net
systemd-resolved[612]: bridge0: Bus client set default route setting: yes
systemd-resolved[612]: bridge0: Bus client set DNS server list to: x.x.x.51, x.x.x.52
NetworkManager[676]: <info>  [1681982583.8464] device (bridge0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
NetworkManager[676]: <info>  [1681982583.8517] device (bridge0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
NetworkManager[676]: <info>  [1681982583.8547] device (bridge0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
NetworkManager[676]: <info>  [1681982583.8574] device (bridge0): Activation: successful, device activated.

Looks like NetworkManager add virtual nic to bridge before physical nic.That explain some of the behavior.

Comment 3 Zbigniew Jędrzejewski-Szmek 2023-04-20 11:21:11 UTC
*full log*, please.

Comment 4 bengt 2023-04-20 11:26:06 UTC
Created attachment 1958541 [details]
requested log

Comment 5 Thomas Haller 2023-04-20 11:43:47 UTC
with MacAddressPolicy=none, the bridge's MAC address is determined by kernel. Which chooses it based on the attached ports, and in particular on the order in which ports get attached.

In your case, the bridge and enp3s0f0 interfaces are configured by NetworkManager, while the vnet[012] interfaces are attached by somebody else (libvirt?)

That somebody is faster than NetworkManager, hence the first interface to be attached is vnet0.

> Predictable mac is used for dhcp request. This was one of main reasons for this change.

What you actually get with MacAddressPolicy=none (and no explicit configuration of the bridge's MAC address otherwise), is that the MAC address of the bridge is *one* of its ports. The distinction whether the port is a virtual nic does not matter to kernel.

Maybe a simple solution is to actually configure the MAC address that you want? Like `nmcli connection modify bridge0 ethernet.cloned-mac-address a8:60:b6:1d:86:17` ?

Comment 6 Thomas Haller 2023-04-20 11:53:12 UTC
it's slightly more involved...

Kernel will assign the "lowest" MAC address to the bridge. This happened, you can see "a8:*" was assigned, which is sorted before the others.

However, before that happened, NetworkManager already started doing DHCP on that interface, at a time when it still had the other MAC address. Hence, the DHCP request uses a different MAC address.


For a short time, there was a patch in NetworkManager to restart DHCP when the MAC address changes ([1]). But that caused other problems (like the IP address changing, when a port gets attached), so it was reverted. I think the solution is still to either configure the bridge's "ethernet.cloned-mac-address" or "ipv4.dhcp-client-id" explicitly.

[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/98197386078a471ed993d009e34e08a6aa7c22d9

Comment 7 Thomas Haller 2023-04-20 11:58:52 UTC
One problem that https://fedoraproject.org/wiki/Changes/MAC_Address_Policy_none addresses, is that when you setup a bridge in your data-center, then the bridge's MAC address and DHCP client ID will be based on *one* of the known, physical MAC addresses of that machine. So you can configure your DHCP server based on that knowledge and based on the inventory of the machine.

It does not solve the problem, that if you don't specify the MAC address, that it will in all cases automagically choose the desired configuration. It seems, you need some explicit configuration here.

Comment 8 bengt 2023-04-20 12:12:40 UTC
`nmcli connection modify bridge0 ethernet.cloned-mac-address a8:60:b6:1d:86:17` mitigate my issue. This will add following to /etc/NetworkManager/system-connections/bridge0.nmconnection

[ethernet]
cloned-mac-address=A8:60:B6:1D:86:17

I read you already have had a long discussion about "MAC Address Policy none". Probably not wise to rehash it.

The bridge was created with cockpit, so it may be good idea to tell the cockpit team to set mac when bridge is created.

It is ok for me to close the ticket, and leave it as a documentation.

Comment 9 Zbigniew Jędrzejewski-Szmek 2023-04-20 13:11:38 UTC
Thank you both for the investigation.
I'd say that this is a poster child for why MACAddressPolicy=persistent was introduced.
But as it was said, we already had a long discussion about this.

Since everything is working as designed, let's close this.


Note You need to log in before you can comment on or make changes to this bug.