Bug 2213200

Description of problem:
We are trying to set up a Macsec tunnel between two servers connected to the same switch using a NetworkManager using this documentation [0].
However, this setup is not passing any traffic. During the debugging and experimenting, we discovered that the configuration only works when the servers are connected back-2-back without a switch. Thus, we suspected the switch configuration, but we could configure a Macsec tunnel manually over the switch using this script [1]. Therefore, we think that the NetworkManager was causing the error.

We are attempting to establish a MACsec tunnel between two servers connected to the same switch using NetworkManager as our configuration tool. Despite following the instructions in the provided documentation [0], the configured setup is not allowing any traffic to pass through the MACsec tunnel.

During the debugging process, we conducted experiments to isolate the issue. Surprisingly, we discovered that the MACsec configuration only works successfully when the servers are directly connected without involving the switch. This led us to suspect that the switch configuration might be causing the problem.

To investigate further, we manually configured a MACsec tunnel over the switch using a custom script [1], which worked as expected. This indicates that the switch itself is not the problem.

Given these findings, we suspect that the issue lies within the NetworkManager.

Version-Release number of selected component (if applicable):
distro: RHEL-9.2.0 GA
kernel: 5.14.0-284.11.1.el9_2.x86_64
NetworkManager: 1.42.2-1.el9.x86_64
wpa_supplicant: 2.10-4.el9.x86_64

How reproducible:
always

Steps to Reproduce:
1. Install a pair of servers connected to the same LAN
2. Pick an interface pair connected to **switch** -> bnx2x_1
3. nmcli connection add type macsec con-name macsec0 ifname macsec0 connection.autoconnect yes macsec.parent bnx2x_1 macsec.mode psk macsec.mka-cak 50b71a8ef0bd5751ea76de6d6c98c03a macsec.mka-ckn f2b4297d39da7330910a74abc0449feb45b5c0b9fc23df1430e1898fcf1c4550 
4. nmcli connection modify macsec0 ipv4.method manual ipv4.addresses '192.0.2.X/24'
5. nmcli connection up macsec0
6. ping 192.0.2.X

Actual results:
Traffic is not passing. The following logs appear in the journalctl log for wpa_supplicant:
Jun 07 14:02:08 kalic1 wpa_supplicant[6733]: KaY: The peer (05018f297c629e59feb179e5) is not my live peer - ignore MACsec SAK Use parameter set
Jun 07 14:02:08 kalic1 wpa_supplicant[6733]: KaY: Discarding Rx MKPDU: decode of parameter set type (3) failed


Expected results:
Traffic passes between interfaces and is encrypted.

Additional info:
We can provide and set up the buggy environment for further investigation.

Links:
[0] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/securing_networks/assembly_using-macsec-to-encrypt-layer-2-traffic-in-the-same-physical-network_securing-networks 

[1] Manual Macsec script:
MAC_01="00:10:18:98:c8:a6"
MAC_02="00:10:18:98:c8:92"

if [ $(hostname) = "kalic1" ]; then
  ip link add link bnx2x_1 macsec0 type macsec encrypt on
  ip macsec add macsec0 tx sa 0 pn 1 on key 00 ead3664f508eb06c40ac7104cdae4ce5
  ip macsec add macsec0 rx port 1 address "$MAC_02"
  ip macsec add macsec0 rx port 1 address "$MAC_02" sa 0 pn 1 on key 01  dffafc8d7b9a43d5b9a3dfbbf6a30c16
  ip l set macsec0 up
  ip a a 192.168.1.1/24 dev macsec0
  echo "done"
fi

if [ $(hostname) = "kalic2" ]; then
  ip link add link bnx2x_1 macsec0 type macsec encrypt on
  ip macsec add macsec0 tx sa 0 pn 1 on key 01 dffafc8d7b9a43d5b9a3dfbbf6a30c16
  ip macsec add macsec0 rx port 1 address "$MAC_01"
  ip macsec add macsec0 rx port 1 address "$MAC_01" sa 0 pn 1 on key 01 ead3664f508eb06c40ac7104cdae4ce5
  ip l set macsec0 up
  ip a a 192.168.1.2/24 dev macsec0
  echo "done"
fi