Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2123413

Summary: persistentnetnamesconfig.py generates broken network naming file when having a bond
Product: Red Hat Enterprise Linux 7 Reporter: Renaud Métrich <rmetrich>
Component: leapp-repositoryAssignee: Leapp Notifications Bot <leapp-notifications-bot>
Status: CLOSED MIGRATED QA Contact: upgrades-and-conversions
Severity: medium Docs Contact:
Priority: medium    
Version: 7.9CC: cbesson, msekleta, pstodulk
Target Milestone: rcKeywords: MigratedToJIRA
Target Release: ---Flags: pm-rhel: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-12 13:21:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Renaud Métrich 2022-09-01 15:25:50 UTC
Description of problem:

When a system has a different naming for network interfaces between RHEL7 and RHEL8, persistentnetnamesconfig.py generates network ".link" files to make sure the RHEL7 naming is kept once upgraded.

The code bases itself on the MAC address of the network interface.
Unfortunately when having a bond, the MAC address for the 2nd slave is not computed correctly: it's the MAC address of the bond (hence 1st slave) that is returned instead of the Permanent MAC address (the MAC address of the physical card).
This generates a broken ".link" file which ends up having the network card be renamed by RHEL8 through using the standard scheme, as shown in the example below:

Interfaces on RHEL7 (all Mellanox cards, which get renamed):

- bond0, MAC 0c:42:a1:1c:b6:d6 with slaves
  - eno1, MAC 0c:42:a1:1c:b6:d6
  - enp59s0f0, MAC 0c:42:a1:24:95:96 but seen as "0c:42:a1:1c:b6:d6" because of the bond
- enp59s0f1, MAC 0c:42:a1:24:95:97

Files created by Leapp for network interfaces:

- /etc/systemd/network/10-leapp-eno1.link (OK)

  [Match]
  MACAddress=0c:42:a1:1c:b6:d6
  [Link]
  Name=eno1

- /etc/systemd/network/10-leapp-enp59s0f0.link (KO because slave of the bond)

  [Match]
  MACAddress=0c:42:a1:1c:b6:d6
  [Link]
  Name=enp59s0f0

- /etc/systemd/network/10-leapp-enp59s0f1.link (OK) 

  [Match]
  MACAddress=0c:42:a1:24:95:97
  [Link]
  Name=enp59s0f1

Interfaces as seen on RHEL8:

- eno1 kept as is
- enp59s0f0 becomes ens1f0 because standard naming is used (/etc/systemd/network/10-leapp-enp59s0f0.link doesn't match because of the MAC address generated in the file)
- enp59s0f1 kept as is because of /etc/systemd/network/10-leapp-enp59s0f1.link

Digging further, it appears the following code is "faulty":

/usr/share/leapp-repository/repositories/system_upgrade/common/libraries/persistentnetnames.py:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  1 import pyudev
 :
 34 def interfaces():
 35     """
 36     Generator which produces an Interface objects containing assorted interface properties relevant for network na    ming
 37     """ 
 38     for dev in physical_interfaces():
 39         attrs = {}
 40 
 41         try:
 42             attrs['name'] = dev.sys_name
 43             attrs['devpath'] = dev.device_path
 44             attrs['driver'] = dev['ID_NET_DRIVER']
 45             attrs['vendor'] = dev['ID_VENDOR_ID']
 46             attrs['pci_info'] = PCIAddress(**pci_info(dev['ID_PATH']))
 47             attrs['mac'] = dev.attributes.get('address')
 :
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

On line 47, the code relies on **address** attribute which is the NON-PERMANENT MAC when part of a bond.

Because udev has no knowledge of the Permanent Address, the only solution I found is to extract the MAC address from the ID_NET_NAME_MAC property instead, something like this patch below:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
# diff -u /usr/share/leapp-repository/repositories/system_upgrade/common/libraries/persistentnetnames.py.orig /usr/share/leapp-repository/repositories/system_upgrade/common/libraries/persistentnetnames.py
--- /usr/share/leapp-repository/repositories/system_upgrade/common/libraries/persistentnetnames.py.orig    2022-08-17 16:57:19.000000000 +0200
+++ /usr/share/leapp-repository/repositories/system_upgrade/common/libraries/persistentnetnames.py    2022-09-01 16:57:58.635000000 +0200
@@ -1,4 +1,5 @@
 import pyudev
+import re
 
 from leapp.libraries.stdlib import api
 from leapp.models import Interface, PCIAddress
@@ -44,9 +45,11 @@
             attrs['driver'] = dev['ID_NET_DRIVER']
             attrs['vendor'] = dev['ID_VENDOR_ID']
             attrs['pci_info'] = PCIAddress(**pci_info(dev['ID_PATH']))
-            attrs['mac'] = dev.attributes.get('address')
-            if isinstance(attrs['mac'], bytes):
-                attrs['mac'] = attrs['mac'].decode()
+            assert dev['ID_NET_NAME_MAC'].startswith('enx')
+            attrs['mac'] = ':'.join(re.findall('..', dev['ID_NET_NAME_MAC'][3:]))
+            #attrs['mac'] = dev.attributes.get('address')
+            #if isinstance(attrs['mac'], bytes):
+            #    attrs['mac'] = attrs['mac'].decode()
         except Exception as e:  # pylint: disable=broad-except
             # FIXME(msekleta): We should probably handle errors more granularly
             # Maybe we should inhibit upgrade process at this point
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

It's ugly/hacky but seems to work but probably DOESN'T cover all cases, specially non-ethernet network interfaces.
Maybe udev needs enhancement to store the permanent address somewhere.

Version-Release number of selected component (if applicable):

leapp-upgrade-el7toel8-0.16.0-8.el7_9.noarch

How reproducible:

Always

Steps to Reproduce:
1. Add 2 network interfaces in a VM and create a bond

  enp7s0: 52:54:00:cf:e9:a8
  enp8s0: 52:54:00:23:46:01

2. Execute "leapp pre-upgrade"
3. Check DB content (/var/lib/leapp/leapp.db)

Actual results: (same MACs for enp7s0 and enp8s0)

{"devpath": "/devices/pci0000:00/0000:00:02.6/0000:07:00.0/net/enp7s0", "driver": "e1000e", "mac": "52:54:00:cf:e9:a8", "name": "enp7s0", "pci_info": {"bus": "07", "device": "00", "domain": "0000", "function": "0"}, "vendor": "0x8086"},
{"devpath": "/devices/pci0000:00/0000:00:02.7/0000:08:00.0/net/enp8s0", "driver": "e1000e", "mac": "52:54:00:cf:e9:a8", "name": "enp8s0", "pci_info": {"bus": "08", "device": "00", "domain": "0000", "function": "0"}, "vendor": "0x8086"}

Expected results: (permanent MAC for enp8s0)

{"devpath": "/devices/pci0000:00/0000:00:02.6/0000:07:00.0/net/enp7s0", "driver": "e1000e", "mac": "52:54:00:cf:e9:a8", "name": "enp7s0", "pci_info": {"bus": "07", "device": "00", "domain": "0000", "function": "0"}, "vendor": "0x8086"},
{"devpath": "/devices/pci0000:00/0000:00:02.7/0000:08:00.0/net/enp8s0", "driver": "e1000e", "mac": "52:54:00:23:46:01", "name": "enp8s0", "pci_info": {"bus": "08", "device": "00", "domain": "0000", "function": "0"}, "vendor": "0x8086"}

Additional info:

I don't know if this applies to Leapp 8 -> 9

Comment 3 Petr Stodulka 2022-09-02 10:34:59 UTC
Hi Renaud! Thank you for the detailed investigation and a proposal for a possible solution which is still improvement even when it probably does not fix all cases. Great job!
This is related to bug 1919382 which led to the current KI & workaround:
  https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/upgrading_from_rhel_7_to_rhel_8/index#known-issues_troubleshooting (search bonding)
The current workaround:
    # LEAPP_NO_NETWORK_RENAMING=1 leapp upgrade

The workaround is not ideal as it skips any actions regarding NIC names, keeping the solution purely on users.
The proposed solution could be an improvement as people could miss the documented workaround prior the upgrade
and also it could enable handling of NIC names in some additional cases with bonding present.


@msekleta : Michal, wdyt about that? Any idea how it could be improved?

Comment 4 Renaud Métrich 2022-09-02 10:58:56 UTC
I think LEAPP_NO_NETWORK_RENAMING=1 cannot be used because this will end up having new RHEL8 names once upgraded, hence break the current ifcfg-XXX files anyway.

Comment 5 Michal Sekletar 2023-02-01 17:50:59 UTC
I think the problem with what Renaud is suggesting is that ID_NET_NAME_MAC don't have to be set. udev's net_id builtin will not set up this variable in case the address which it finds is not permanent (addr_assign_type != 0). Hence if MAC is changed and then something (tool/admin) calls "udevadm trigger" then ID_NET_NAME_MAC will be dropped.

Maybe we should also take into account value of addr_assign_type and if the address isn't permanent we wouldn't use it, instead we could fallback to "ethtool -P" as that should return true hardware address. All of that has a problem that there might be cases when MAC addresses are changed on purpose and not because of bonds and such. We may want to introduce configuration option for this and set it to false by default. Here is a pseudo-code of new logic that we might introduce,

if has_permanent_mac(iface)
  use(iface.mac)
else if is_bond_slave_or_similar(iface)  // are there other cases except bonding? 
  use(ethtool_fallback(iface))
else if config_use_virtual_macs(config)
  use(iface.mac)
else
  log_and_skip(iface)

Btw, this split through the cracks terribly, you guys have to annoy me more if I don't reply in reasonable time!

Comment 6 RHEL Program Management 2023-09-12 12:24:51 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 7 RHEL Program Management 2023-09-12 13:21:13 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.