As part of ocp single node installation effort we saw that sometimes etc/resolv.conf has no data. 
We didn't see such failures while using ocp 4.8.fc3.
We wonder if something was changed that could explain this new issue?

Currently we are trying to workaround the problem with https://github.com/openshift/assisted-service/pull/1971

Thomas, is this the same as bug 1778798 ?

(In reply to Till Maas from comment #3)
> Thomas, is this the same as bug 1778798 ?

yes, that is possible.



The attached log is only at info level, it shows however as first thing:

 sno00476 NetworkManager[1629]: <info>  [1623367432.9500] device (enp1s0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
 sno00476 NetworkManager[1629]: <warn>  [1623367433.0477] dispatcher: (2) /etc/NetworkManager/dispatcher.d/forcedns failed (failed): Script '/etc/NetworkManager/dispatcher.d/forcedns'...

this happens log before an interface is activated, so it would be expected that resolv.conf is still empty at that point.



But without debugging logfiles (or a patch to test) that is hard to asses.

Given that it's only happening 1.5% of the times, it might be infeasible to collect level=TRACE logs. But if possible, that would be very helpful to see the log for such a boot. Please see https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/blob/bae22a45d837e76e805f0f411b2d71748e76625e/contrib/fedora/rpm/NetworkManager.conf#L27 for various hints about logging of NetworkManager. Also, don't only provide `journalctl -u NetworkManager`. Please provide complete logs that at least also include the logging of NetworkManager-dispatcher.service.





---

Sidenote:

The dispatcher script messing with /etc/resolv.conf while NetworkManager also writes to that file is inherently race (even without any bug in NetworkManager).

If you have a (dispatcher) script that writes /etc/resolv.conf, then you possibly should ensure that only your script is doing that. You can drop a `rc-manager=unmanaged` setting to /{etc,run}/NetworkManager/conf.d (and reload the configuration with `systemctl reload NetworkManager` or `killall -HUP NetworkManager`). Also, NetworkManager defaults to `rc-manager=symlink`, which means, if you replace /etc/resolv.conf with a symlink, then NetworkManager will leave the file alone. See `man NetworkManager.conf`. So an alternative script might get the information from `/run/NetworkManager/resolv.conf` (but don't modify NM's file!), then writes it to /etc/resolv.conf.suffix and make a symlink `ln -snf resolv.conf.suffix /etc/resolv.conf`. There is still a tiny race the very first time when your tool creates the symlink (if at that time NetworkManager saw the file earlier, decides to (atomically) rewrite the file, and then replaces the symlink you just wrote). One problem is what happens after reboot. Is it OK that the symlink is still there? Depends...

You can either have a simple or a correct script. But not both. The question is how correct vs. how simple it needs to be.

Hi Igal,

I recall openshift AI made some changes on this DNS setup for this confliction on dispatch script and NetworkManager. Does it solve the problem?

@fge yep, changes to the script fixed the issue for us.