Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
As part of ocp single node installation effort we saw that sometimes etc/resolv.conf has no data.
We didn't see such failures while using ocp 4.8.fc3.
We wonder if something was changed that could explain this new issue?
(In reply to Till Maas from comment #3)
> Thomas, is this the same as bug 1778798 ?
yes, that is possible.
The attached log is only at info level, it shows however as first thing:
sno00476 NetworkManager[1629]: <info> [1623367432.9500] device (enp1s0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
sno00476 NetworkManager[1629]: <warn> [1623367433.0477] dispatcher: (2) /etc/NetworkManager/dispatcher.d/forcedns failed (failed): Script '/etc/NetworkManager/dispatcher.d/forcedns'...
this happens log before an interface is activated, so it would be expected that resolv.conf is still empty at that point.
But without debugging logfiles (or a patch to test) that is hard to asses.
Given that it's only happening 1.5% of the times, it might be infeasible to collect level=TRACE logs. But if possible, that would be very helpful to see the log for such a boot. Please see https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/blob/bae22a45d837e76e805f0f411b2d71748e76625e/contrib/fedora/rpm/NetworkManager.conf#L27 for various hints about logging of NetworkManager. Also, don't only provide `journalctl -u NetworkManager`. Please provide complete logs that at least also include the logging of NetworkManager-dispatcher.service.
---
Sidenote:
The dispatcher script messing with /etc/resolv.conf while NetworkManager also writes to that file is inherently race (even without any bug in NetworkManager).
If you have a (dispatcher) script that writes /etc/resolv.conf, then you possibly should ensure that only your script is doing that. You can drop a `rc-manager=unmanaged` setting to /{etc,run}/NetworkManager/conf.d (and reload the configuration with `systemctl reload NetworkManager` or `killall -HUP NetworkManager`). Also, NetworkManager defaults to `rc-manager=symlink`, which means, if you replace /etc/resolv.conf with a symlink, then NetworkManager will leave the file alone. See `man NetworkManager.conf`. So an alternative script might get the information from `/run/NetworkManager/resolv.conf` (but don't modify NM's file!), then writes it to /etc/resolv.conf.suffix and make a symlink `ln -snf resolv.conf.suffix /etc/resolv.conf`. There is still a tiny race the very first time when your tool creates the symlink (if at that time NetworkManager saw the file earlier, decides to (atomically) rewrite the file, and then replaces the symlink you just wrote). One problem is what happens after reboot. Is it OK that the symlink is still there? Depends...
You can either have a simple or a correct script. But not both. The question is how correct vs. how simple it needs to be.
Hi Igal,
I recall openshift AI made some changes on this DNS setup for this confliction on dispatch script and NetworkManager. Does it solve the problem?