Bug 1778798

Summary: [NMCI] nmcli reporting activated but nameservers are not written yet
Product: Red Hat Enterprise Linux 8 Reporter: Vladimir Benes <vbenes>
Component: NetworkManagerAssignee: Thomas Haller <thaller>
Status: CLOSED WORKSFORME QA Contact: Vladimir Benes <vbenes>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.1CC: atragler, bgalvani, fge, jmaxwell, lrintel, pasik, rkhan, sukulkar, thaller, till
Target Milestone: rcKeywords: Triaged
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-25 13:32:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vladimir Benes 2019-12-02 14:12:10 UTC
Description of problem:
sometimes I see an issue with nameservers not yet written into resolv.conf and activated state being reported by nmcli. This should be fixed.

Version-Release number of selected component (if applicable):
1.22

How reproducible:
ipv4_dns_resolvconf_symlinked test

Additional info:
https://desktopqe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/beaker-NetworkManager-master-default-rhel8-upstream/924/artifact/artifacts/FAIL_report_NetworkManager-ci_Test307_ipv4_dns_resolvconf_symlinked.html

these are not valid as I see:
[root@wsfd-netdev34-vm-3 NetworkManager-ci]# cat /tmp/resolv.conf
# Generated by NetworkManager
search ntdv.lab.eng.bos.redhat.com
nameserver 10.19.42.41
nameserver 10.11.5.19
nameserver 10.5.30.160
[root@wsfd-netdev34-vm-3 NetworkManager-ci]# cat /tmp/resolv_orig.conf
# Generated by NetworkManager

Comment 1 Beniamino Galvani 2020-03-06 13:53:20 UTC
It seems important to have this fixed in RHEL 8.3.

Comment 4 Thomas Haller 2021-06-25 13:32:58 UTC
Since then, the test ipv4_dns_resolvconf_symlinked passes.

At the time of reporting, Vladimir also did:

 https://gitlab.freedesktop.org/NetworkManager/NetworkManager-ci/-/commit/1a5da593bd7a366d351b710d2e130e9eed6c8d02

so presumably that patch necessary back then to workaround this issue.




I tried now to restore the old test (from 1a5da593bd7~1). I couldn't reproduce the problem.



I also tried to change

  When "nameserver" is visible with command "cat /etc/resolv.conf" in "20" seconds

to

  When "nameserver" is visible with command "cat /etc/resolv.conf" in "0" seconds

(which requires https://gitlab.freedesktop.org/NetworkManager/NetworkManager-ci/-/merge_requests/786).
That also didn't reproduce the issue.



So that means, the original problem is no longer reproducible, at least as far as I can tell.
I recommend to update the tests @ipv4_dns_resolvconf_symlinked and @ipv4_dns_resolvconf_file to
drop the workaround: https://gitlab.freedesktop.org/NetworkManager/NetworkManager-ci/-/merge_requests/788




In general, I think that there are some subtle issues with ordering from activated and DNS configuration.
For example, when using systemd-resolved, then DNS configuration is done asynchronously and in the background.
That means, when NM declares a profile to be activated, DNS might not yet be configured (that should be fixed).

But this bug is not about using systemd-resolved, but plain /etc/resolv.conf. In that case, NMPolicy subscribes to
NM_DEVICE_STATE_CHANGED signal before updating DNS. However, that signal is emitted before after `_notify(self, PROP_STATE);`,
so on D-Bus signal is already sent out (and theoretically could reach the client first). So there is a race, and
there is a potential problem here.


However, I would rather address this problem by the Layer3Config work and following.
Also, a theoretical issue that has no clear reproducer, is hard to address.

This rhbz was reported with a CI test failure. The CI no longer fails.
Presumably something changed in the past year.


If you still can reproduce, please reopen.

Comment 5 Thomas Haller 2021-06-25 13:35:26 UTC
(In reply to Thomas Haller from comment #4)

@vbenes FYI ^^