Bug 1868315

Summary:	CoreDNS pods observing panic without any provbe failures
Product:	OpenShift Container Platform	Reporter:	Apoorva Jagtap <apjagtap>
Component:	Networking	Assignee:	Stephen Greene <sgreene>
Networking sub component:	DNS	QA Contact:	Hongan Li <hongli>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	unspecified	CC:	aos-bugs, apjagtap, dhansen, eparis, mjoseph, scott.worthington
Version:	4.4	Keywords:	FeatureBackport
Target Milestone:	---
Target Release:	4.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Running CoreDNS 1.6.6 Consequence: Intermittent `invalid memory address or nil pointer dereference` followed by timeouts in kube API access Fix: Backport CoreDNS#4022 to OCP, which fixes handling errors with Endpoint Tombstones. Result: CoreDNS behaves as intended without intermittent panics.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 16:28:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1869309

Description Apoorva Jagtap 2020-08-12 09:50:12 UTC

Description of problem:
Encountering 'Observed a panic' error messages in the logs of CoreDns pods.

Version-Release number of selected component (if applicable):
- RHOCP v4.4.9
- CoreDNS version 1.6.6

How reproducible:
- The scenario is observed in CU's environment momentarily. 
- Not sure whether it could be reproduced or not.

Actual results:
Encountering 'Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)' messages in the logs of some CoreDNS pods, without any liveness/readiness failure, leading to DNS failures in the cluster.

Expected results:
- This bug has been reported in the upstream: https://github.com/coredns/coredns/issues/4022
- And the fix is available for CoreDNS v1.7.0: https://github.com/coredns/coredns/pull/3887

- Would it possible to backport it to CoreDNS v1.6.6?

Comment 7 Hongan Li 2020-08-19 07:42:32 UTC

verified with 4.6.0-0.nightly-2020-08-17-150352	and no issue/panic during the regression test.

Comment 9 errata-xmlrpc 2020-10-27 16:28:06 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196