Bug 1548102 - [3.7] A route with host 'localhost' can freeze router reloads under some circumstances
Summary: [3.7] A route with host 'localhost' can freeze router reloads under some cir...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.7.0
Hardware: All
OS: All
unspecified
medium
Target Milestone: ---
: 3.7.z
Assignee: Ravi Sankar
QA Contact: zhaozhanqi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-22 17:47 UTC by Ravi Sankar
Modified: 2022-08-04 22:20 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: A user can set a hostname to localhost, which confuses the router check to see if the reload completed. Consequence: The reload never returns success causing the router to fail. Fix: Change the health check so that it does not pass hostname 'localhost'. Result: Router reloads successfully.
Clone Of:
Environment:
Last Closed: 2018-04-29 14:36:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:1231 0 None None None 2018-04-29 14:37:19 UTC

Description Ravi Sankar 2018-02-22 17:47:11 UTC
Description of problem:
We have found out that creating a route with host 'localhost' can freeze subsequent router reloads IF AND ONLY IF the environment variable ROUTER_USE_PROXY_PROTOCOL is not set.

Version-Release number of selected component (if applicable):
v3.6.173.0.96

How reproducible:
always if ROUTER_USE_PROXY_PROTOCOL is not set

Steps to Reproduce:
- Ensure your router does not have ROUTER_USE_PROXY_PROTOCOL environment variable set
- Create a route with 'localhost' hostname that can be admitted on routers. For instance, the one issued by this command: oc create route edge localhost-route --hostname=localhost --service=internetcheck --insecure-policy=Allow
- Try to create/delete other routes so that a reload is forced. You will see errors and reloads will fail.

Actual results:

Errors seen at logs are like the following:

E0131 16:00:58.114791       1 ratelimiter.go:52] error reloading router: exit status 1
[WARNING] 030/160028 (360) : parsing [/var/lib/haproxy/conf/haproxy.config:35] : 'option httplog' not usable with frontend 'public_ssl' (needs 'mode http'). Falling back to 'option tcplog'.
 - Checking http://localhost:2080 ...
 - Exceeded max wait time (30) in health check - 58 retry attempt(s).

Expected results:
Something (safety checks?) to prevent this situation from happening.

Additional info:
3.9 bug: https://bugzilla.redhat.com/show_bug.cgi?id=1542612

Comment 1 Ravi Sankar 2018-02-22 18:11:04 UTC
Fixed by https://github.com/openshift/ose/pull/1078

Comment 2 Alejandro Coma 2018-03-22 12:09:35 UTC
Hi Ravi,

The pull URL is not working for me, can you please give me an update on the status of this?

Thanks!

Comment 3 Ravi Sankar 2018-03-22 18:58:47 UTC
This is the actual Origin PR https://github.com/openshift/origin/pull/18539 that got back ported to 3.8 and 3.7 releases (you should be able to access this pr). 
3.7 back-port https://github.com/openshift/ose/pull/1078 got merged and should be available in the next 3.7.z release.

Comment 5 zhaozhanqi 2018-04-17 02:20:37 UTC
Verified this bug on v3.7.44

steps:

Create service:
#oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/unsecure/service_unsecure.json
#oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/caddy-docker.json
#oc create route edge localhost-route --hostname=localhost --service=service-unsecure --insecure-policy=Allow

#oc expose svc service-unsecure

###check the last route can be accessed and make the router has been reloaded

# check the router logs and no above logs

Comment 7 zhaozhanqi 2018-04-23 02:22:07 UTC
Verified this bug, see comment 5 for more detail

Comment 11 errata-xmlrpc 2018-04-29 14:36:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1231


Note You need to log in before you can comment on or make changes to this bug.