Bug 1534816 - Path based routing is broken for mixed tls
Summary: Path based routing is broken for mixed tls
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Routing
Version: 3.9.0
Hardware: All
OS: Linux
high
high
Target Milestone: ---
: 3.10.0
Assignee: jtanenba
QA Contact: zhaozhanqi
URL:
Whiteboard:
Depends On: 1534488
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-16 02:32 UTC by zhaozhanqi
Modified: 2018-10-03 16:11 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Splitting up the route types into separate map files Consequence: Because haproxy looks for the first match and a route with no path is a catch all for all other paths would cause haproxy to match the wrong route with some configurations Fix: Merge maps that make sense and make sure that they are searched appropriately Result: Correctly match the incoming requests with the corresponding backends
Clone Of: 1534488
Environment:
Last Closed: 2018-10-03 16:11:19 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Origin (Github) 15847 None None None 2018-01-24 15:26:18 UTC

Description zhaozhanqi 2018-01-16 02:32:03 UTC
+++ This bug was initially created as a clone of Bug #1534488 +++

Description of problem:
If you mix 2 Routes (edge terminated) one with InsecureEdgeTerminationPolicy Redirect and the other one with Allow, path based routing doesn't work and the redirect Route always wins.

Version-Release number of selected component (if applicable):
All current versions are broken.

How reproducible:
Always.

Actual results:
All traffic goes to Redirect Route.

Expected results:
Traffic goes by the rules of path based routing as declared in: 
https://docs.openshift.org/latest/architecture/networking/routes.html#path-based-routes


Additional info:
related issues:
 - https://github.com/openshift/origin/issues/14950
 - https://github.com/tnozicka/openshift-acme/issues/16


There is already a pull from community to fix it 

  https://github.com/openshift/origin/pull/15847

waiting for a review from networking team since Aug 18, 2017.

Comment 2 zhaozhanqi 2018-01-25 08:24:16 UTC
After testing this, found the issue still cannot be fixed yet

Tested this on version
oc v3.9.0-0.23.0
kubernetes v1.9.1+a0ce1bc657
features: Basic-Auth GSSAPI Kerberos SPNEGO

steps:

1. Create pod1/svc/route
  $oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/networking/list_for_pods.json
  $ oc create route edge edge1 --service=test-service --insecure-policy=Allow

2. Create another pod2/svc and with same route hostname
  $ oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/caddy-docker.json
  $ oc create -f oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/unsecure/service_unsecure.json
  $ oc expose svc service-unsecure --hostname='edge1-z1.apps.0124-uvc.qe.rhcloud.com' --path=/test

3. Check the route
  # oc get route
NAME               HOST/PORT                               PATH      SERVICES           PORT      TERMINATION   WILDCARD
edge1              edge1-z1.apps.0124-uvc.qe.rhcloud.com             test-service       http      edge/Allow    None
service-unsecure   edge1-z1.apps.0124-uvc.qe.rhcloud.com   /test     service-unsecure   http                    None
 
4. When I access the route w and w/o path one,they are both refer to pod1
  $ # curl edge1-z1.apps.0124-uvc.qe.rhcloud.com
Hello OpenShift!
  $ curl edge1-z1.apps.0124-uvc.qe.rhcloud.com/test/
Hello OpenShift!

5. let's delete the 'edge1' route to prove the route with path one can work 
  $ oc delete route edge1
  $ oc get route
NAME               HOST/PORT                               PATH      SERVICES           PORT      TERMINATION   WILDCARD
service-unsecure   edge1-z1.apps.0124-uvc.qe.rhcloud.com   /test     service-unsecure   http                    None
$ curl edge1-z1.apps.0124-uvc.qe.rhcloud.com/test/
Hello-OpenShift-Path-Test http-8080

Comment 4 jtanenba 2018-01-31 18:33:34 UTC
Could you test with a router built from the source tree?

Comment 5 zhaozhanqi 2018-02-01 03:09:59 UTC
@jtanehba

I'm not sure why we need to rebuilt the router from the source tree..

since I already checked the our testing env router and the related fixed PR https://github.com/openshift/origin/pull/15847 already in.

I can give you the env if you need

Comment 6 zhaozhanqi 2018-02-01 05:03:41 UTC
more info:

$ oc get route
NAME               HOST/PORT                               PATH      SERVICES           PORT      TERMINATION   WILDCARD
edge1              edge1-z1.apps.0201-8nl.qe.rhcloud.com             test-service       http      edge/Allow    None
service-unsecure   edge1-z1.apps.0201-8nl.qe.rhcloud.com   /test     service-unsecure   http                    None

*************Test the route**************
$ curl edge1-z1.apps.0201-8nl.qe.rhcloud.com
Hello OpenShift!

$ curl edge1-z1.apps.0201-8nl.qe.rhcloud.com/test/
Hello OpenShift!

**************Check the map file****************
$ cat os_http_be.map 
^edge1-z1\.apps\.0201-8nl\.qe\.rhcloud\.com(:[0-9]+)?/test(/.*)?$ z1:service-unsecure

sh-4.2$ cat os_edge_http_be.map 
^edge1-z1\.apps\.0201-8nl\.qe\.rhcloud\.com(:[0-9]+)?(/.*)?$ z1:edge1

sh-4.2$ cat os_route_http_insecure.map
^edge1-z1\.apps\.0201-8nl\.qe\.rhcloud\.com(:[0-9]+)?(/.*)?$ be_edge_http:z1:edge1

***************************

From the content of the map file and haproxy.config. I guess 'edge1-z1.apps.0201-8nl.qe.rhcloud.com/test/' should also can be matched toos_route_http_insecure.map with high priority than os_http_be.map

Comment 7 jtanenba 2018-02-02 19:04:53 UTC
Could you give me access to the env?

Comment 8 jtanenba 2018-02-02 21:26:52 UTC
Upon further investigation it is found that an issue with the map files match incorrectly and a different setup of map files needs to exist to correctly match the backends

Comment 9 Tomáš Nožička 2018-02-06 17:59:28 UTC
Can QA please verify that it also works correctly in environments where the Router is backed by something else than HA proxy, like F5?

Comment 10 Ben Bennett 2018-02-15 17:18:48 UTC
This behavior has not changed from earlier releases... so I think we need to revert the earlier attempt at a fix, and then we need to consider what we expect to happen here.

My advice is not to mix different termination types for the same host at this point.  We'll see if we can make the router handle different terminations or reject routes if they don't match.  But that will be a later release.

Comment 11 Tomáš Nožička 2018-02-16 08:21:04 UTC
bbennett@redhat.com please note that it's desired to mix the termination types -here is a use case: User has a regular Route with insecureEdgeTerminationPolicy=(None,Redirect) and he needs to redirect traffic for ACME verification as a subpath (/.well-known/...) with insecureEdgeTerminationPolicy==Allow.

(We'll soon to have a controller in the OpenShift org using just that which is kind of hacky to work around this bug.)

Comment 12 openshift-github-bot 2018-03-28 15:19:08 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/8d5346ea3e189502bd3f95ccdfa4c570729409fa
Combine backend map files to fix bath based routing

combined the two maps for insecure routes os_edge_http_be.map and os_route_http_expose.map and the two maps for secure routes os_reencrypt.map and os_edge_http_be.map  reducing the number of map files and fixing path based routing

Bug 1534816

Comment 13 zhaozhanqi 2018-04-13 08:52:02 UTC
Verified this bug on v3.10.0-0.15.0 with haproxy images id (5bcd6b8168cc) using comment 6 steps.


Note You need to log in before you can comment on or make changes to this bug.