Hide Forgot
Description of problem: Using the doc: https://docs.openshift.com/container-platform/3.11/install_config/router/default_haproxy_router.html#exposing-the-router-metrics, to enable router metrics page for browser. Would cause the pod to fail to start up with "Readiness probe failed: HTTP probe failed with statuscode: 401". Version-Release number of selected component (if applicable): 3.11.51 How reproducible: - Repeatedly. Steps to Reproduce: 1. A working cluster with router pods 2. oc set env dc router ROUTER_METRICS_TYPE- ROUTER_LISTEN_ADDR- 3. oc -n default get pod -w - shows pod stuck at deploy and eventually fails deploy 4. oc -n default get events - shows readiness probe failure Actual results: - router pod fails to deploy Expected results: - router pod deploy successful and metrics page enabled for browser at port 1936 Additional info:
Workaround based on what Luke found in the case 02284589 from 3.9: - oc patch dc router -p '"spec": {"template": {"spec": {"containers": [{"name": "router","readinessProbe": {"httpGet": {"path": "healthz"}}}]}}}' - oc set env dc router-dmz ROUTER_METRICS_TYPE-
(In reply to Alan C from comment #1) > Workaround based on what Luke found in the case 02284589 from 3.9: > > - oc patch dc router -p '"spec": {"template": {"spec": {"containers": > [{"name": "router","readinessProbe": {"httpGet": {"path": "healthz"}}}]}}}' > > - oc set env dc router-dmz ROUTER_METRICS_TYPE- Sorry, typo, should be: - oc set env dc router ROUTER_METRICS_TYPE- ...to match the patch command.
Yeah, the issue is we changed the probe paths to be different to distinguish between liveness and readiness checks. But when you disable the listener on the stats port on the openshift-router and enable it on haproxy, there is only one "unauthenticated" endpoint available (as haproxy only allows one monitor-uri) to use. So we do have to match up both the probes to the same endpoint "/healthz" when we remove ROUTER_LISTEN_ADDR - will create a PR for the docs team to also mention that we need to update the readiness probe in that section. Thanks.
Docs PR: https://github.com/openshift/openshift-docs/pull/13608
Moving to MODIFIED as the docs update merged.
Docs PR against master: https://github.com/openshift/openshift-docs/pull/13622 The 3.11 PR (https://github.com/openshift/openshift-docs/pull/13608) got closed by Vikram so am not certain if this is ready for QA unless I missed another docs PR.
the Doc PR looks good, thanks
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0407