Bug 1589740

Summary: Panic logs shown when enable ROUTER_BIND_PORTS_AFTER_SYNC
Product: OpenShift Container Platform Reporter: zhaozhanqi <zzhao>
Component: NetworkingAssignee: Ben Bennett <bbennett>
Networking sub component: router QA Contact: zhaozhanqi <zzhao>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: aos-bugs, bbennett, hongli
Version: 3.10.0   
Target Milestone: ---   
Target Release: 3.10.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-30 19:17:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zhaozhanqi 2018-06-11 10:03:15 UTC
Description of problem:
When set env ROUTER_BIND_PORTS_AFTER_SYNC to true for router. the router will show logs:

    1 logs.go:49] http: panic serving [::1]:43816: runtime error: invalid memory address or nil pointer dereference
goroutine 144 [running]:
net/http.(*conn).serve.func1(0xc420de21e0)
        /usr/lib/golang/src/net/http/server.go:1697 +0xd0
panic(0x4fda380, 0xac11af0)
        /usr/lib/golang/src/runtime/panic.go:491 +0x283
github.com/openshift/origin/pkg/router/metrics.HasSynced.func1(0xc421266700, 0x0, 0x0)
        /builddir/build/BUILD/atomic-openshift-git-0.8285be6/_output/local/go/src/github.com/openshift/origin/pkg/router/metrics/health.go:44 +0x31
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz.(*healthzCheck).Check(0xc42050a5c0, 0xc421266700, 0x5aab3ca, 0x1e)
        /builddir/build/BUILD/atomic-openshift-git-0.8285be6/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:109 +0x34
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz.handleRootHealthz.func1(0xac9fe60, 0xc420362fc0, 0xc421266700)
        /builddir/build/BUILD/atomic-openshift-git-0.8285be6/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:118 +0xdc
net/http.HandlerFunc.ServeHTTP(0xc4203df3a0, 0xac9fe60, 0xc420362fc0, 0xc421266700)
        /usr/lib/golang/src/net/http/server.go:1918 +0x44
net/http.(*ServeMux).ServeHTTP(0xc4201589c0, 0xac9fe60, 0xc420362fc0, 0xc421266700)
        /usr/lib/golang/src/net/http/server.go:2254 +0x130
net/http.serverHandler.ServeHTTP(0xc420f44270, 0xac9fe60, 0xc420362fc0, 0xc421266700)
        /usr/lib/golang/src/net/http/server.go:2619 +0xb4
net/http.(*conn).serve(0xc420de21e0, 0xaca40e0, 0xc421254540)
        /usr/lib/golang/src/net/http/server.go:1801 +0x71d
created by net/http.(*Server).Serve
        /usr/lib/golang/src/net/http/server.go:2720 +0x288

Version-Release number of selected component (if applicable):
oc v3.10.0-0.64.0
kubernetes v1.10.0+b81c8f8
features: Basic-Auth GSSAPI Kerberos SPNEGO

crio version 1.10.2-dev

How reproducible:
always

Steps to Reproduce:
1. set the env for router
   oc env dc router ROUTER_BIND_PORTS_AFTER_SYNC=true
2. Check the logs of router
3.

Actual results:

see panic logs in $description

Expected results:

router can work well and no this logs 

Additional info:

Comment 1 Ben Bennett 2018-06-11 14:45:13 UTC
Fixed by https://github.com/openshift/origin/pull/19959

Comment 2 openshift-github-bot 2018-06-11 21:42:25 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/ef2847bdce883de725846aaded7b17803a4930fe
Prevent a router panic when ROUTER_BIND_PORTS_AFTER_SYNC is set

The function that was checking whether the router had synced needed to check the nilness of the target of a pointer, not the pointer itself.

I also corrected the logic so that when the routerPtr target was not defined, it says the router has not synced.

Fixes bug 1589740 (https://bugzilla.redhat.com/show_bug.cgi?id=1589740)

Comment 4 zhaozhanqi 2018-06-14 02:49:54 UTC
Verified this bug on v3.10.0-0.67.0. this bug has been fixed.

Comment 6 errata-xmlrpc 2018-07-30 19:17:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816