We should not forward on PTR requests for either pod CIDR or services CIDR ranges. If a service ip comes and goes it doesn't make sense to forward that request on and for pod ips we should determined if we want to provide PTR records or simply return negative results for all requests. This stems from the troubleshooting in https://bugzilla.redhat.com/show_bug.cgi?id=1420121 where it was determined that pods were doing reverse lookups for the host's pod CIDR ip address. This was a 3.4 environment but it was hoped that the chagnes in 3.6 would've fixed this but that's not the case.
Scott, not sure if you meant to assign this to Pod component, but seems that this is either Networking or Installer related. I would guess Installer (responsible for dnsmasq configuration) but can you move this to one of those two components?
Seth, It'd be whichever component is responsible for running the dns service embedded in the node. Networking seems fine.
I need someone to help me figure out whether the DNS spec allows a forwarding resolver to fail a PTR request - if so we can fix this by carrying a patch on skydns that allows us to reject (SERVFAIL) the PTR. However, may also need some help determining whether SERVFAIL is allowed for PTR responses.
Assuming the cluster is actually using private address ranges, then RFC 6761 "Special-Use Domain Names" lets us return any answer we want here: 6.1. Domain Name Reservation Considerations for Private Addresses The private-address [RFC1918] reverse-mapping domains listed below, and any names falling within those domains, are Special-Use Domain Names: 10.in-addr.arpa. 21.172.in-addr.arpa. 26.172.in-addr.arpa. 16.172.in-addr.arpa. 22.172.in-addr.arpa. 27.172.in-addr.arpa. 17.172.in-addr.arpa. 30.172.in-addr.arpa. 28.172.in-addr.arpa. ... These domains, and any names falling within these domains, are special in the following ways: .... 4. Caching DNS servers SHOULD recognize these names as special and SHOULD NOT, by default, attempt to look up NS records for them, or otherwise query authoritative DNS servers in an attempt to resolve these names. Instead, caching DNS servers SHOULD, by default, generate immediate (positive or negative) responses for all such queries. This is to avoid unnecessary load on the root name servers and other name servers. Caching DNS servers SHOULD offer a configuration option (disabled by default) to enable upstream resolution of such names, for use in private networks where private-address reverse-mapping names are known to be handled by an authoritative DNS server in said private network. Even in the non-private-IP-range case, a non-authoritative resolver is still always allowed to return any error except NXDOMAIN (and all of the error codes apply to all query types). So yes, you could return SERVFAIL, although REFUSED would be better; SERVFAIL ("The name server was unable to process this query due to a problem with the name server") is sometimes treated as a temporary/retryable error (eg, getnameinfo() returns EAI_AGAIN and net.LookupAddr() returns an error with .IsTemporary == true). REFUSED ("The name server refuses to perform the specified operation for policy reasons") generally gets treated by clients the same as "not found".
Is there any trello card, git hub, or work on this that can be tracked? There was discussion on this previously and then it was put on "UpcomingRelease" pointing to 3.7, then 3.8, and now 3.9.
With 3.9 node-dnsmasq.conf has: -- server=/in-addr.arpa/127.0.0.1 server=/cluster.local/127.0.0.1 -- So it sends all reverse lookups to our local resolver. Our resolver now returns useful answers for services: -- $ nslookup 172.30.0.1 172.30.0.1 Server: 172.30.0.1 Address: 172.30.0.1#53 Non-authoritative answer: 1.0.30.172.in-addr.arpa name = kubernetes.default.svc.cluster.local. Authoritative answers can be found from: -- But for a pod IP address it returns SERVFAIL which makes dnsmasq try the upstream servers. However, that puts us at the mercy of the upstream resolver to be quick enough, and if we get a resonse, we will definitely get an incorrect name. We could hack this by adding a line to the node-dnsmasq.conf in ansible that tells dnsmasq to handle the appropriate set of in-addr.arpa itself. That would look something like: local=/130.10.in-addr.arpa/ (see http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html) The issue there is that it only works when the SDN CIDR is 8-bit aligned. So if you have a SDN CIDR that is /15 we will reply for more than we should :-(
@Eric: What problem is the customer still experiencing? The service CIDRs are being resolved quickly. Is the upstream resolver being slow when resolving pod IPs? When I tested 1000 resolutions, it took ~6 seconds regardless of whether I was testing a pod or service address against our resolver directly, or whether I was using a good address against the upstream resolver.