Red Hat Bugzilla – Bug 1401425
[3.4]The nameservers in /etc/resolv.conf are generated to non-node's ip by NetworkManager after rebooting nodes
Last modified: 2017-03-08 13:43 EST
Description of problem: After rebooting the node which belongs to the OCP cluster, the nameserver in /etc/resolv.conf would be regenerated automatically by NetworkManager. Actually we expect it should still be the node's ip Version-Release number of selected component (if applicable): openshift-ansible-3.4.33-1 NetworkManager-1.4.0-13.el7_3.x86_64 How reproducible: always Steps to Reproduce: 1. Install OCP 2. Check /etc/resolv.conf on each node 3. reboot one node after installation 4. Check /etc/resolv.conf on each node Actual results: 2. # cat /etc/resolv.conf # Generated by NetworkManager search c.openshift-gce-devel.internal google.internal nameserver 10.240.0.35 --> the node's ip # nameserver updated by /etc/NetworkManager/dispatcher.d/99-origin-dns.sh 4. # cat /etc/resolv.conf # Generated by NetworkManager search c.openshift-gce-devel.internal google.internal nameserver 169.254.169.254 Expected results: Additional info: Restart NetworkManager can fix it. It looks like NetworkManager starts before dnsmasq on a reboot. #journalctl -u NetworkManager Dec 05 03:33:00 qe-ghuang-conrhel-master-1 NetworkManager[662]: <info> [1480926780.5415] device (eth0): Activation: successful, device activated. Dec 05 03:33:04 qe-ghuang-conrhel-master-1 NetworkManager[662]: <info> [1480926784.9640] manager: startup complete #journalctl -u dnsmasq -l -- Logs begin at Mon 2016-12-05 03:32:44 EST, end at Mon 2016-12-05 03:47:05 EST. -- Dec 05 03:33:05 qe-ghuang-conrhel-master-1 systemd[1]: Started DNS caching server.. Dec 05 03:33:05 qe-ghuang-conrhel-master-1 systemd[1]: Starting DNS caching server....
Hmm, dispatcher script should start it if it's not started. Will look into this today.
This is a regression introduced in https://github.com/openshift/openshift-ansible/pull/2690
Proposed fix https://github.com/openshift/openshift-ansible/pull/2915
Commit pushed to master at https://github.com/openshift/openshift-ansible https://github.com/openshift/openshift-ansible/commit/7d9b624ea0707eaf6875a6781aed2f7f387fe349 node_dnsmasq - restart dnsmasq if it's not currently running Fixes Bug 1401425 Fixes BZ1401425
Verified with origin/release-1.4 branch Will test again once new puddle coming
Verified this bug with openshift-ansible-3.4.35-1.git.0.2e13650.el7, and PASS. After reboot node, the nameserver in /etc/resolv.conf is still kept to node's IP.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0066