Bug 2213258 - NetworkManager MACSEC on a bond device - wobbly
Summary: NetworkManager MACSEC on a bond device - wobbly
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: NetworkManager
Version: CentOS Stream
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: NetworkManager Development Team
QA Contact: Desktop QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-07 16:26 UTC by lejeczek
Modified: 2023-07-19 10:14 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-07-07 13:35:06 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
debug NM boot log (55.01 KB, application/x-7z-compressed)
2023-07-08 08:34 UTC, lejeczek
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker NMT-602 0 None None None 2023-06-07 16:33:48 UTC
Red Hat Issue Tracker RHELPLAN-159204 0 None None None 2023-06-07 16:33:53 UTC

Description lejeczek 2023-06-07 16:26:34 UTC
Description of problem:

Hi guys, perhaps not a bug but future enhancement suggestion.
I have a MACSEC iface created off a 'bond' device, which bond is in 'broadcast' mode & is too managed by NM.
Connection is set with:
...
connection.interface-name:              macsec0
...
results in: (bare in mind only a single 'macsec' iface exists configured in NM)

-> $ ip macsec sh
8: macsec0: protect off validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off 
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: ee7e1fe4a2440001 on SA 0
    offload: off 
14: macsec1: protect on validate strict sc on sa on encrypt on send_sci on end_station off scb off replay off 
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 8cdcd4aae03c0001 on SA 2
        2: PN 4, state on, key bf...
    RXSC: a2c33455508c0001, state on
        2: PN 5403, state on, key bf..
    RXSC: 00110a6bf7b40001, state on

At which point - naturally - there is no connection to the peers.
I can do:
-> $ _CON=macsec-10.1.1-br; nmcli c d $_CON ; nmcli c u $_CON
and that "fixes" the issue:
-> $ ip macsec sh
16: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off 
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 8cdcd4aae03c0001 on SA 0
    RXSC: 00110a6bf7b40001, state on
    RXSC: a2c33455508c0001, state on
    offload: off 

I think you get the gist of it - I presume there is a "randomness" to how NM stand up devices(stack) &| how system enumerates physical devices or more..

I'm fiddling with it "this" way, as I understand NM/kernel/drivers &| more cannot enslave a 'macsec' device - as of now - at least it's what NM tells me.

Here is 'nmcli' for macsec:
-> $ nmcli c add type macsec con-name macsec-10.1.1-br ifname macsec0 connection.autoconnect yes macsec.parent bond-1011 macsec.mode psk macsec.mka-cak d9... macsec.mka-ckn 7a... ipv4.method disabled ipv6.method disabled con.slave-type bridge con.master 99...

If..
-> $ nmcli c m macsec-10.1.1-br con.interface-name macsec1
then (after reboot)
8: macsec1: protect off validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off 
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 0279de0480af0001 on SA 0
    offload: off 
14: macsec0: protect on validate strict sc on sa on encrypt on send_sci on end_station off scb off replay off 
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 8cdcd4aae03c0001 on SA 2
        2: PN 12, state on, key 0ef9..
    RXSC: 00110a6aba140001, state on
        2: PN 1584, state on, key 0ef9..
    RXSC: 00110a6bf7b40001, state on
        2: PN 3204, state on, key 0ef92..
    offload: off 


Version-Release number of selected component (if applicable):

NetworkManager-1.43.8-1.el9.x86_64
NetworkManager-libnm-1.43.8-1.el9.x86_64
NetworkManager-team-1.43.8-1.el9.x86_64

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Thomas Haller 2023-06-07 19:42:37 UTC
I don't understand what happens.

Are you saying, that you have one macsec profile in NetworkManager, but see two macsec profiles in ip-link? Who creates the other one?

You say, the wrong thing is present after reboot. Please enable `level=TRACE` logging as described in the DEBUGGING section in `man NetworkManager`. Then reboot and reproduce. Then provide the complete journal (of that boot). Also show the output of `nmcli connection`, `nmcli device`, and `ip -d link` and `ip macsec show`. And, for the relevant profiles also show `nmcli -o connection show "$PROFILE_NAME"`.

Thank you.

> bare in mind only a single 'macsec' iface exists configured in NM

Btw, in NetworkManager you configure connection profiles (short "connections"), and by activating them the interface will be created by NetworkManager. It seems less confusing, if you think about having profiles, and what happens when you activate them, and which profiles are currently activated.

Comment 2 lejeczek 2023-06-08 07:09:06 UTC
I'll try my best to get that additional info but I the mean while, it should be trivial to reproduce at least certain bits:

a) I have a 2-port 10Gb NIC
b) I make a bond connection off such NIC, in "broadcast" mode
c) I make a MACSEC off that "bond" device/connection
d) lastly (probably optional & redundant - might as well add IP bits to the "bond" device/connection & end there though I've not tried) hook such MACSEC connection into a "bridge" connection/profile, which bridge has all IP bits set up for net communication.

All above is done with/under NM & as I described earlier.

Yes - I end up having two MACSEC devices (show up in: "ip macsec") unless ! I do "down & up" on connection/profile (as described above) but that "fixes" the issue only until next reboot, which reboot brings back macsecs 0 & 1 again.... "Who creates the other one?" - exactly!
I'ld only swap "who" for "what".

I failed to find a work-around with/in NM for TWO macseces after reboot issue - so I took a "poor man" route to work around this and I tell cron to "down & up" connection \@reboot.

This is a physical set-up I'm doing this on, but who knows, perhaps even in a KVM it might reproduce.

Comment 3 Thomas Haller 2023-06-08 07:43:51 UTC
It may be trivial to reproduce, when the involved configuration is clearly shown. But "make a MACSEC" is not a precise description of how to get there. What *exact* profiles did you configure?

Is there a problem with attaching the requested information from comment 1? Please attach it, the logfile is important. Thank you.

Comment 4 lejeczek 2023-06-08 08:04:35 UTC
I showed that in my first, original comment.
Here:
-> $ nmcli c add type macsec con-name macsec-10.1.1-br ifname macsec0 connection.autoconnect yes macsec.parent bond-1011 macsec.mode psk macsec.mka-cak d9... macsec.mka-ckn 7a... ipv4.method disabled ipv6.method disabled con.slave-type bridge con.master bridge-conn

I said, I'll try my best - I feel like I have the state less obvious: I'm trying to make living and I do that by having "up & running relatively okey computer systems" - in other words: cannot afford stop, tear down, restart &| debug at whim all day long. Give me a while please.

Comment 5 Thomas Haller 2023-06-08 09:54:07 UTC
the shown `nmcli c add` line creates only one macsec profile. For this setup to work, also the exact bond, bridge an bond-ports are relevant. I can make them up, and when I activate my "reproducer" a macsec0 interface is created. There is no macsec1 interface, and obviously it cannot come out of nowhere.

If you cannot reboot your system with debug logs enabled and reproduce the issue, then please provide the information that you have right at hand. As explained in comment 1.

Check all the profiles you have (`nmcli connection`). Ensure there is no profile that would create a macsec1 interface (by either deleting it, or at least ensure that it's not activated). See activated devices and profiles with `nmcli device`. And show that output here.

Comment 6 sfaye 2023-07-07 13:35:06 UTC
Hi, 

We cannot move this bug forward without the requested information in comment 5. Therefore, we will close this as INSUFFICIENT DATA for now. Please feel free to reopen it when you have the information we need. 

Thanks

Comment 7 lejeczek 2023-07-08 08:34:27 UTC
Created attachment 1974626 [details]
debug NM boot log

Comment 8 lejeczek 2023-07-17 11:12:39 UTC
Nothing, no good, NM log I attached?


Note You need to log in before you can comment on or make changes to this bug.