Bug 1402488 - After namespace router shard implementation router intermittently does not get the route
Summary: After namespace router shard implementation router intermittently does not ge...
Keywords:
Status: CLOSED DUPLICATE of bug 1383663
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Ram Ranganathan
QA Contact: zhaozhanqi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-07 16:22 UTC by Steven Walter
Modified: 2022-08-04 22:20 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-27 16:25:03 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Steven Walter 2016-12-07 16:22:57 UTC
Description of problem:
We have implemented router shard namespaces in our automation lab. During testing of this change we found that intermittently that the router does not pick up the route. In some cases it is enough to re-create the route and in the worse case I need to restart the router pod for the route to be added to `/var/lib/haproxy/router/routes.json`.

Version-Release number of selected component (if applicable):
3.3.0

How reproducible:
Unconfirmed

Steps to Reproduce:
Putting in next comment



Actual results:
$ oc exec ${INTERNAL_ROUTER} -n default -- bash -c 'cat /var/lib/haproxy/router/routes.json' | grep internal-router-test
$ oc exec ${DMZ_ROUTER} -n default -- bash -c 'cat /var/lib/haproxy/router/routes.json' | grep internal-router-test
$

Expected results:
$ INTERNAL_ROUTER=$(oc get pods -l router=router --no-headers=true -n default | awk '{print $1}')
$ DMZ_ROUTER=$(oc get pods -l router=router-dmz --no-headers=true -n default | awk '{print $1}')
$ oc exec ${INTERNAL_ROUTER} -n default -- bash -c 'cat /var/lib/haproxy/router/routes.json' | grep internal-router-test
    "internal-router-test_php": {
      "Namespace": "internal-router-test",
      "Host": "php-internal-router-test.paas-a.am.lilly.com",
        "internal-router-test/php": 100
$ oc exec ${DMZ_ROUTER} -n default -- bash -c 'cat /var/lib/haproxy/router/routes.json' | grep internal-router-test
$

Comment 2 Steven Walter 2016-12-07 16:24:33 UTC
After waiting the 10 minute refresh interval, the routes still did not show up.

Router logs just show:

    I1205 12:00:14.036891       1 router.go:404] Router reloaded:
     - Checking HAProxy /healthz on port 1936 ...
     - HAProxy port 1936 health check ok : 0 retry attempt(s).
    I1205 12:00:21.950161       1 router.go:404] Router reloaded:
     - Checking HAProxy /healthz on port 1936 ...
     - HAProxy port 1936 health check ok : 0 retry attempt(s).
    I1205 12:00:32.653189       1 router.go:404] Router reloaded:
     - Checking HAProxy /healthz on port 1936 ...
     - HAProxy port 1936 health check ok : 0 retry attempt(s).
    I1205 12:00:33.593595       1 router.go:404] Router reloaded:
     - Checking HAProxy /healthz on port 1936 ...
     - HAProxy port 1936 health check ok : 0 retry attempt(s).
    I1205 12:04:04.020082       1 router.go:404] Router reloaded:
     - Checking HAProxy /healthz on port 1936 ...
     - HAProxy port 1936 health check ok : 0 retry attempt(s).

etc

Comment 5 zhaozhanqi 2016-12-08 07:12:32 UTC
hi, Steven

From the comment 1

<--snip-->

 oc exec ${INTERNAL_ROUTER} -n default -- bash -c 'cat /var/lib/haproxy/router/routes.json' | grep internal-router-test
    "internal-router-test_php": {
      "Namespace": "internal-router-test",
      "Host": "php-internal-router-test.paas-a.am.lilly.com",
        "internal-router-test/php": 100
  [14:14:39] u3x1609:~ $ oc exec ${DMZ_ROUTER} -n default -- bash -c 'cat /var/lib/haproxy/router/routes.json' | grep internal-router-test

<--snip-->

I think this is correct result since ${DMZ_ROUTER} router only match label dmz-router=yes"

Comment 6 Steven Walter 2016-12-08 14:49:09 UTC
Hi,

Yes, that is what they are hoping for. That output is correct. However they only see it after manually restarting the router. It is supposed to show up in the ${INTERNAL_ROUTER}, but sometimes when they create the route it does not show in the ${INTERNAL_ROUTER}, and they have to delete the pod (creates a new pod) to have it show up inside the router.

Comment 8 Ben Bennett 2017-01-27 16:25:03 UTC

*** This bug has been marked as a duplicate of bug 1383663 ***


Note You need to log in before you can comment on or make changes to this bug.