Bug 1401427

Summary:	[3.3]The nameservers in /etc/resolv.conf are generated to non-node's ip by NetworkManager after rebooting nodes
Product:	OpenShift Container Platform	Reporter:	Gan Huang <ghuang>
Component:	Installer	Assignee:	Scott Dodson <sdodson>
Status:	CLOSED ERRATA	QA Contact:	Johnny Liu <jialiu>
Severity:	high	Docs Contact:
Priority:	medium
Version:	3.3.1	CC:	aos-bugs, jialiu, jokerman, mmccomas, sdodson
Target Milestone:	---	Keywords:	Reopened
Target Release:	3.3.1
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Previously the NetworkManager dispatcher script did not correctly update /etc/resolv.conf after a host was rebooted. The script has been updated to ensure that /etc/resolv.conf is updated on reboot ensuring proper use of dnsmasq.	Story Points:	---
Clone Of:	1401425	Environment:
Last Closed:	2017-01-24 21:09:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1401425, 1401428
Bug Blocks:

Description Gan Huang 2016-12-05 09:00:15 UTC

+++ This bug was initially created as a clone of Bug #1401425 +++

Description of problem:
After rebooting the node which belongs to the OCP cluster, the nameserver in /etc/resolv.conf would be regenerated automatically by NetworkManager. Actually we expect it should still be the node's ip

Version-Release number of selected component (if applicable):
openshift-ansible-3.4.33-1
NetworkManager-1.4.0-13.el7_3.x86_64

How reproducible:
always

Steps to Reproduce:
1. Install OCP
2. Check /etc/resolv.conf on each node
3. reboot one node after installation
4. Check /etc/resolv.conf on each node

Actual results:

2. 
# cat /etc/resolv.conf 
# Generated by NetworkManager
search c.openshift-gce-devel.internal google.internal
nameserver 10.240.0.35   --> the node's ip
# nameserver updated by /etc/NetworkManager/dispatcher.d/99-origin-dns.sh

4. 
# cat /etc/resolv.conf 
# Generated by NetworkManager
search c.openshift-gce-devel.internal google.internal
nameserver 169.254.169.254

Expected results:


Additional info:

Restart NetworkManager can fix it.

It looks like NetworkManager starts before dnsmasq on a reboot.

#journalctl -u NetworkManager
Dec 05 03:33:00 qe-ghuang-conrhel-master-1 NetworkManager[662]: <info>  [1480926780.5415] device (eth0): Activation: successful, device activated.
Dec 05 03:33:04 qe-ghuang-conrhel-master-1 NetworkManager[662]: <info>  [1480926784.9640] manager: startup complete

#journalctl -u dnsmasq -l
-- Logs begin at Mon 2016-12-05 03:32:44 EST, end at Mon 2016-12-05 03:47:05 EST. --
Dec 05 03:33:05 qe-ghuang-conrhel-master-1 systemd[1]: Started DNS caching server..
Dec 05 03:33:05 qe-ghuang-conrhel-master-1 systemd[1]: Starting DNS caching server....

Comment 2 Johnny Liu 2016-12-08 09:51:08 UTC

Verified this bug with openshift-ansible-3.3.57-1.git.0.f275588.el7.noarch, and PASS.

After reboot node, the nameserver in /etc/resolv.conf is still kept to node's IP.

Comment 4 errata-xmlrpc 2017-01-24 21:09:45 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0189