Bug 1582818 - HAProxy router causes panic due to index out of range in cmux library
Summary: HAProxy router causes panic due to index out of range in cmux library
Keywords:
Status: CLOSED DUPLICATE of bug 1532060
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Ben Bennett
QA Contact: Meng Bo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-27 04:48 UTC by Kenjiro Nakayama
Modified: 2018-05-29 14:14 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-29 14:14:45 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
oc get dc router -o yaml (4.55 KB, text/plain)
2018-05-27 04:50 UTC, Kenjiro Nakayama
no flags Details

Description Kenjiro Nakayama 2018-05-27 04:48:34 UTC
Description of problem:

- After installed OCP and deployed haproxy router v3.7, HAProxy router causes panic and restarting.

Version-Release number of selected component (if applicable):
- atomic-openshift-3.7.44-2.git.0.6b061d4.el7.x86_64
- w/ HAProxy image v3.7.44 and v3.7.46 also happened.

  image: "openshift3/ose-haproxy-router:v3.7.44",
  image: openshift3/ose-haproxy-router:v3.7.46

How reproducible:
- 100% on only Customer's env

Steps to Reproduce:
1. Install OpenShift v3.7
2. Wait for some hours or days

Actual results:
- Start panicking with following errors:

  # oc logs -p router-1-5k2s6
  I0522 23:52:45.621933       1 template.go:254] Starting template router (v3.7.46)
  I0522 23:52:45.734512       1 metrics.go:148] Router health and metrics port listening at 0.0.0.0:1936
  I0522 23:52:47.311747       1 router.go:441] Router reloaded:
   - Checking http://localhost:80 ...
   - Health check ok : 0 retry attempt(s).
  I0522 23:52:47.311877       1 router.go:228] Router is including routes in all namespaces
  I0522 23:52:47.679454       1 router.go:441] Router reloaded:
   - Checking http://localhost:80 ...
   - Health check ok : 0 retry attempt(s).
  I0522 23:52:52.692311       1 router.go:441] Router reloaded:
   - Checking http://localhost:80 ...
   - Health check ok : 0 retry attempt(s).
  W0523 00:00:49.389745       1 reflector.go:343] github.com/openshift/origin/pkg/router/controller/factory/factory.go:108: watch of *route.Route ended with: etcdserver: mvcc: required revision has been compacted
  panic: runtime error: index out of range
  goroutine 1633 [running]:
  github.com/openshift/origin/vendor/github.com/cockroachdb/cmux.(*ptNode).match(0xc4211c5c20, 0xc4211bbd98, 0x0, 0x8, 0x1, 0x0)
          /builddir/build/BUILD/atomic-openshift-git-0.e81594b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/cockroachdb/cmux/patricia.go:148 +0x197
  github.com/openshift/origin/vendor/github.com/cockroachdb/cmux.(*patriciaTree).matchPrefix(0xc4211cb9c0, 0xf2f2600, 0xc420e7e118, 0xf30b700)
          /builddir/build/BUILD/atomic-openshift-git-0.e81594b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/cockroachdb/cmux/patricia.go:38 +0x90
  github.com/openshift/origin/vendor/github.com/cockroachdb/cmux.(*patriciaTree).(github.com/openshift/origin/vendor/github.com/cockroachdb/cmux.matchPrefix)-fm(0xf2f2600, 0xc420e7e118, 0xc42095e398)
          /builddir/build/BUILD/atomic-openshift-git-0.e81594b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/cockroachdb/cmux/matchers.go:23 +0x3e
  github.com/openshift/origin/vendor/github.com/cockroachdb/cmux.(*cMux).serve(0xc4211a9ec0, 0xf365a80, 0xc42095e398, 0xc42119dce0, 0xc42103baa0)
          /builddir/build/BUILD/atomic-openshift-git-0.e81594b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/cockroachdb/cmux/cmux.go:129 +0x265
  created by github.com/openshift/origin/vendor/github.com/cockroachdb/cmux.(*cMux).Serve
          /builddir/build/BUILD/atomic-openshift-git-0.e81594b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/cockroachdb/cmux/cmux.go:119 +0x16c

Expected results:
- No panic

Additional info:
- Attached installtion steps and DC of router

Comment 2 Kenjiro Nakayama 2018-05-27 04:50:29 UTC
Created attachment 1442188 [details]
oc get dc router  -o yaml

Comment 4 Ben Bennett 2018-05-29 14:14:45 UTC

*** This bug has been marked as a duplicate of bug 1532060 ***


Note You need to log in before you can comment on or make changes to this bug.