RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2150000 - infinite autoconnect retries with multi-connect profiles leaves installations stuck at "A start job is running for nm-wait-online-initrd.service" [rhel-9]
Summary: infinite autoconnect retries with multi-connect profiles leaves installations...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: NetworkManager
Version: 9.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: NetworkManager Development Team
QA Contact: Filip Pokryvka
URL:
Whiteboard:
: 2143978 (view as bug list)
Depends On:
Blocks: 2155531
TreeView+ depends on / blocked
 
Reported: 2022-12-01 14:57 UTC by Martin Hoyer
Modified: 2023-05-09 10:23 UTC (History)
11 users (show)

Fixed In Version: NetworkManager-1.41.7-2.el9
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 2155531 (view as bug list)
Environment:
Last Closed: 2023-05-09 08:17:33 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker NMT-263 0 None None None 2023-02-08 08:41:19 UTC
Red Hat Issue Tracker RHELPLAN-141047 0 None None None 2022-12-01 15:00:52 UTC
Red Hat Product Errata RHBA-2023:2485 0 None None None 2023-05-09 08:17:52 UTC
freedesktop.org Gitlab NetworkManager NetworkManager-ci merge_requests 1282 0 None opened connection: add connection_multiconnect_autoconnect_retries test 2023-01-09 22:07:31 UTC

Description Martin Hoyer 2022-12-01 14:57:12 UTC 
Description of problem:
When trying to install RHEL-9.2 from PXE, after booting to dracut it gets stuck on 'A start job is running for nm-wait-online-initrd.service' 


[  OK  ] Finished dracut pre-trigger hook.
         Starting Coldplug All udev Devices...
[  OK  ] Finished Coldplug All udev Devices.
         Starting nm-initrd.service...
         Starting Show Plymouth Boot Screen...
         Starting Wait for udev To …plete Device Initialization... 
         Starting D-Bus System Message Bus... 
[  OK  ] Started  D-Bus System Message Bus. 
[  OK  ] Started  nm-initrd.service. 
[  OK  ] Reached  target  Network. 
         Starting nm-wait-online-initrd.service... 
[  OK  ] Started  Show Plymouth Boot Screen. 
[  OK  ] Started         
[    5.538872] nvme nvme0: missing or invalid SUBNQN field. 
Forward Password R…s to Plymouth Directory Watch. 
[  OK  ] Reached target Local Encrypted Volumes. 
[  OK  ] Reached target Path Units. 
[ *    ] A start job is running for nm-wait-…ine-initrd.service (5s / no limit) 
^^this line repeats

The machines affected have one ethernet interface as default route, also used for the PXE installation and two network devices that are in different network, which does not have DHCP.
They would normally get the static IP set in anaconda from kickstart, but it will never get there.

It seems to only affect some machines.
Sometimes it proceeds after ~1.5 minutes

Version-Release number of selected component (if applicable):
RHEL-9.2.0-20221129.2 
dracut-057-13.git20220816.el9
NetworkManager-1.41.5-1.el9

Hard to pinpoint where it started doing this, but I believe I haven't encountered this in 9.1

How reproducible:
often

Not sure if nm is at fault here, please re-assign as needed.
Comment 1 Beniamino Galvani 2022-12-01 16:11:51 UTC 
Please add `rd.debug` to kernel command line, reproduce and and attach the console output.
Comment 3 Beniamino Galvani 2022-12-05 13:28:54 UTC 
The problem is that the activation on enp35s0f0 fails and is retried indefinitely (instead of just one time).

I think this is related to the changes in bug 2039734, see comment 5 there.
Comment 4 Martin Hoyer 2022-12-05 14:28:02 UTC 
(In reply to Beniamino Galvani from comment #3)
> I think this is related to the changes in bug 2039734, see comment 5 there.

Indeed, thanks for looking into it!
Comment 5 Thomas Haller 2022-12-20 17:26:58 UTC 
the problematic patch has been reverted upstream:
https://bugzilla.redhat.com/show_bug.cgi?id=2039734#c14

once that hits rhel8/rhel9, the immediate problem caused by the attempt to fix bug 2039734 will be fixed.
Comment 6 Thomas Haller 2022-12-20 17:29:09 UTC 
*** Bug 2143978 has been marked as a duplicate of this bug. ***
Comment 9 PaulB 2023-01-05 14:49:40 UTC 
All,
Looks like the issue is fixed with latest NetworkManager package.



------------------------------------------------------------
I see the fix is in NetworkManager-1.41.7-2.el9.aarch64.rpm:
------------------------------------------------------------
$ rpm -qp --changelog http://download-node-02.eng.bos.redhat.com/brewroot/packages/NetworkManager/1.41.7/2.el9/aarch64/NetworkManager-1.41.7-2.el9.aarch64.rpm | more
---%<-snip->%---
* Wed Dec 21 2022 Thomas Haller <thaller> - 1:1.41.7-2
- core: avoid infinite autoconnect with multi-connect profiles (rh #2150000)
---%<-snip->%---


---------------------------------------
NetworkManager-1.41.6-1.el9.aarch64.rpm 
---------------------------------------
host: ampere-mtjade
distro: RHEL-9.2.0-20221219.0
NetworkManager: 1.41.6-1.el9.aarch64.rpm
https://beaker.engineering.redhat.com/recipes/13132015 - fail

host: ampere-mtcollins
distro: RHEL-9.2.0-20221219.0
NetworkManager: 1.41.6-1.el9.aarch64.rpm
https://beaker.engineering.redhat.com/recipes/13132017 - fail


---------------------------------------
NetworkManager-1.41.7-2.el9.aarch64.rpm 
---------------------------------------
host: ampere-mtjade
distro: RHEL-9.2.0-20221224.0
NetworkManager: 1.41.7-2.el9.aarch64.rpm
https://beaker.engineering.redhat.com/recipes/13175368 - pass


host: ampere-mtcollins
distro: RHEL-9.2.0-20221224.0
NetworkManager: 1.41.7-2.el9.aarch64.rpm
https://beaker.engineering.redhat.com/recipes/13175370 - pass



---------
Notably:
---------
There is still a  ~90seconds  delay for nm-wait-online-initrd.service.
(maybe that is expected?)

https://beaker.engineering.redhat.com/recipes/13175370
https://beaker-archive.hosts.prod.psi.bos.redhat.com/beaker-logs/2023/01/73930/7393010/13175370/console.log
---%<-snip->%---
*      
] A start job is running for nm-wait-…ine-initrd.service (9s / no limit)  

---%<-snip->%---

*       
] A start job is running for nm-wait-…itrd.service (1min 34s / no limit)  
 M   
[      
  OK     
] Finished         
nm-wait-online-initrd.service  

---%<-snip->%---

Best,
pbunyan
Comment 11 errata-xmlrpc 2023-05-09 08:17:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (NetworkManager bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2485
Note You need to log in before you can comment on or make changes to this bug.