Bug 1955632

Summary: CoreDNS resolution failure for external hostnames with "A: dns: overflow unpacking uint16"
Product: OpenShift Container Platform Reporter: Stephen Greene <sgreene>
Component: NetworkingAssignee: Stephen Greene <sgreene>
Networking sub component: DNS QA Contact: jechen <jechen>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: amcdermo, aos-bugs, arghosh, dahernan, fgiloux, hongli, mapandey, mjoseph, palonsor, rcernin, rsales, sgreene
Version: 4.5   
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Cluster upstream resolver returns DNS response that exceeds 512 bytes via UDP. Consequence: CoreDNS may return SERVFAIL and or log various error messages, sometimes forcing the client to retry over TCP. Fix: Enable the CoreDNS bufisze plugin with a UDP buffer size of 1232 bytes. Result: CoreDNS is less likely to return SERVFAIL or present any runtime errors when handling large DNS responses via UDP. Also, UDP packet fragmentation is less likely to occur.
 Story Points: ---
Clone Of: 1953097 Environment:
Last Closed: 2021-05-26 06:27:55 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1953097    
Bug Blocks:    

Comment 2 jechen 2021-05-16 22:15:20 UTC 
Verified in 4.6.0-0.nightly-2021-05-14-175757

$ oc get clusterversions.config.openshift.io 
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2021-05-14-175757   True        False         32m     Cluster version is 4.6.0-0.nightly-2021-05-14-175757


# verified bufsize in default DNS
$ oc -n openshift-dns get cm/dns-default -oyaml
apiVersion: v1
data:
  Corefile: |
    .:5353 {
        bufsize 1232    <--verified the fix by https://github.com/openshift/cluster-dns-operator/pull/272
        errors
        health {
            lameduck 20s
        }


# created a test pod as custom DNS server, added custom forwarding into coreDNS, verified bufsize in custom DNS
$ oc create -f https://raw.githubusercontent.com/lihongan/test-dnsmasq/master/test-dnsmasq.json 
pod/test-dnsmasq created


$ oc get pod -owide
NAME           READY   STATUS    RESTARTS   AGE   IP            NODE                                       NOMINATED NODE   READINESS GATES
test-dnsmasq   1/1     Running   0          27s   10.129.2.15   ci-ln-f245bgt-f76d1-ghn6t-worker-c-2hkjt   <none>           <none>


$ oc edit dns.operator/default 
<--snip-->
spec:
  servers:
  - forwardPlugin:
      upstreams:
      - 10.129.2.15
    name: test
    zones:
    - mytest.ocp
<--snip-->


$ oc get cm dns-default -n openshift-dns -o yaml
apiVersion: v1
data:
  Corefile: |
    # test
    mytest.ocp:5353 {
        forward . 10.129.2.15
        errors
        bufsize 1232   <--verified the fix by https://github.com/openshift/cluster-dns-operator/pull/272
    }
    .:5353 {
        bufsize 1232   <--verified the fix by https://github.com/openshift/cluster-dns-operator/pull/272
        errors
        health {
            lameduck 20s
        }
<--snip-->
Comment 5 errata-xmlrpc 2021-05-26 06:27:55 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.30 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1565