Bug 1673026

Summary: Kubernetes ExternalName Services that point to in-cluster resources broken
Product: OpenShift Container Platform Reporter: Ben Browning <bbrownin>
Component: NetworkingAssignee: Dan Mace <dmace>
Networking sub component: router QA Contact: Hongan Li <hongli>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: aos-bugs
Version: 4.1.0Keywords: NeedsTestCase
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:42:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ben Browning 2019-02-06 13:42:44 UTC
Description of problem:

No Kubernetes ExternalName Services can be resolved via DNS when they point to in-cluster resources.

An example of the kind of ExternalName Service that this breaks:

apiVersion: v1
kind: Service
metadata:
  name: external-internal
spec:
  externalName: image-registry.openshift-image-registry.svc.cluster.local
  type: ExternalName


This is an upstream bug in CoreDNS, which OpenShift moved to in 4.0.

Upstream bug report - https://github.com/coredns/coredns/issues/2038

Upstream fix - https://github.com/coredns/coredns/pull/2040

Pull request to move us to a newer version of CoreDNS with this fix in our openshift/coredns fork - https://github.com/openshift/coredns/pull/6


Version-Release number of selected component (if applicable):

Anything in 4.0



How reproducible:

This can be reproduced every time.


Steps to Reproduce:

# Create two ExternalName services - one pointing to a domain
# inside the cluster and one pointing outside the cluster
cat <<EOF | oc apply -f -
---
apiVersion: v1
kind: Service
metadata:
  name: external-internal
spec:
  externalName: image-registry.openshift-image-registry.svc.cluster.local
  type: ExternalName
---
apiVersion: v1
kind: Service
metadata:
  name: external-external
spec:
  externalName: redhat.com
  type: ExternalName
EOF

# Start a busybox pod so we can test DNS resolution
oc run busybox -it --rm=true --restart=Never --image=busybox

# From inside the running pod, try pinging a few addresses

# PASS: resolve the ExternalName service that points to an external domain
ping -c 1 external-external | grep PING
PING external-external (209.132.183.105): 56 data bytes

# FAIL: resolve the ExternalName service that points to an internal domain
ping -c 1 external-internal | grep PING
ping: bad address 'external-internal'

# PASS: resolve the internal domain that the external-internal service points to
ping -c 1 image-registry.openshift-image-registry.svc.cluster.local | grep PING
PING image-registry.openshift-image-registry.svc.cluster.local (172.30.3.178): 56 data bytes


Actual results:

The external-internal domain cannot resolve. That is the ExternalName service that points to an in-cluster resource.


Expected results:

The external-internal domain should resolve. I've verified that when running a newer CoreDNS inside OpenShift 4 (built from the openshift/coredns PR linked above), it does resolve.

Comment 2 Hongan Li 2019-02-19 03:24:20 UTC
verified with 4.0.0-0.nightly-2019-02-17-024922 and issue has been fixed.

$ oc exec dns-default-64v7s -c dns -- coredns -version
CoreDNS-1.3.1
linux/amd64, go1.10.8, 

$ oc rsh centos-pod 
sh-4.2$ 
sh-4.2$ nslookup external-external
Server:		172.30.0.10
Address:	172.30.0.10#53

external-external.hongli.svc.cluster.local	canonical name = redhat.com.
Name:	redhat.com
Address: 209.132.183.105

sh-4.2$ 
sh-4.2$ nslookup external-internal
Server:		172.30.0.10
Address:	172.30.0.10#53

external-internal.hongli.svc.cluster.local	canonical name = image-registry.openshift-image-registry.svc.cluster.local.
Name:	image-registry.openshift-image-registry.svc.cluster.local
Address: 172.30.174.31

Comment 5 errata-xmlrpc 2019-06-04 10:42:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758