Bug 1261736

Summary: containers running on node can not resolve internal service name when setting up native master HA using haproxy as LB
Product: OpenShift Container Platform Reporter: Johnny Liu <jialiu>
Component: InstallerAssignee: Andrew Butcher <abutcher>
Status: CLOSED CURRENTRELEASE QA Contact: Ma xiaoqiang <xiama>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0.0CC: aos-bugs, bleanhar, bperkins, dmcphers, jokerman, mmccomas, xtian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-20 15:42:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Johnny Liu 2015-09-10 05:45:13 UTC 
Description of problem:
Use https://github.com/detiber/openshift-ansible -b next-ha to set up a mutli-masters HA env. 

In the env, a haproxy LB will take over master api role as frontend which is listening at 8443, the real masters will be backend of the LB. 

In some use cases, containers running on nodes need resolve internal service name, skydns (which is running together with master api) could resolve these internal service names, "dnsIP" setting in node yaml config file should be pointed to the skydns IP so that containers know where to resolve them. 

In current installation, the "dnsIP" is being pointed to the LB's IP, it is not real master, there is no skydns service is running, so containers running on nodes can not resolve service name.



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 2 Brenton Leanhardt 2015-09-30 13:27:30 UTC 
This is related to https://bugzilla.redhat.com/show_bug.cgi?id=1242961#c9 and the work Andrew is doing to have resolv.conf on the Nodes point to the kubernetes Service.
Comment 3 Andrew Butcher 2015-11-02 16:00:06 UTC 
Node configuration will use the master service address as the node dns ip.

https://github.com/openshift/openshift-ansible/pull/608
Comment 4 Johnny Liu 2015-11-03 10:38:18 UTC 
Verified this bug with AtomicOpenShift/3.1/2015-11-02.1 puddle and https://github.com/abutcher/openshift-ansible -b native-ha, PASS.


# oc get svc kubernetes
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)                 SELECTOR   AGE
kubernetes   172.30.0.1   <none>        443/TCP,53/UDP,53/TCP   <none>     4h


# grep dnsIP /etc/origin/node/node-config.yaml 
dnsIP: 172.30.0.1


# docker exec -ti 09744e8e6f2b /bin/sh
sh-4.2$ cat /etc/resolv.conf 
nameserver 172.30.0.1
nameserver 10.72.17.5
search jialiu.svc.cluster.local svc.cluster.local cluster.local openstacklocal lab.eng.nay.redhat.com
options ndots:5
sh-4.2$ curl simple-openshift-sinatra:8080
Hello, Sinatra!