Bug 1691190 - passthrough route created using path [NEEDINFO]
Summary: passthrough route created using path
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Routing
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.3.0
Assignee: Miciah Dashiel Butler Masters
QA Contact: Hongan Li
URL:
Whiteboard:
Depends On:
Blocks: 1884422
TreeView+ depends on / blocked
 
Reported: 2019-03-21 05:24 UTC by Ashish Prajapati
Modified: 2020-11-17 07:22 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1884422 (view as bug list)
Environment:
Last Closed: 2020-01-23 11:03:45 UTC
Target Upstream Version:
aprajapa: needinfo? (dmace)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift router pull 57 0 'None' closed Bug 1691190: hostindex: Passthrough displaces path-based TLS routes 2021-02-11 22:35:35 UTC
Github openshift router pull 59 0 'None' closed Bug 1691190: hostindex: Path-based TLS displaces passthrough 2021-02-11 22:35:35 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:04:13 UTC

Description Ashish Prajapati 2019-03-21 05:24:07 UTC
Description of problem:
Passthrough routes does not support path based routing. 
Why this route can be created?
It should throw an error.

But actual result is Route gets created and that is unusable.

Replicte
1st scenario:

Create a pass-through route with --path variable and it gets created without an error.

2nd Scenario:

Create a simple pass-through route.
Create one more pass-through route with same hostname as above and give a path to the route.
It also gets created.

Actual results:
it is processed same as other routes.

Expected results:
In  both cases, It should throw an error, While creating pass-through route it should not check for hostname+path combination, and should throw an error stating not supported.

Comment 1 Ashish Prajapati 2019-03-22 05:41:39 UTC
Hello Team,

I would like to add one more point which I missed in description for the scenario 2, In which we are creating a new route with hostname already being used by a passthrough route, when you also specify a path for that new route, it gets created without a error.

Actual Results:

The duplicate domain detection does not recognize the domain part of this new route when specified a path to the new route.

Expected:

It should throw an error and the duplicate domain detection should not treat the passthrough route as other routes.


Thanks.

Comment 2 Keno Engelbracht 2019-03-22 16:59:16 UTC
Hi,
to amend the initial description:


The problem occurs when edge terminated route that specifies a path is created after a passthrough route using the same domain. Only then it is not detected by the domain collision prevention, because the check only looks for routes with the same "path" parameter, which the algorithm considers to be "/" for a passthrough route instead of a wildcard which would would prevent any additional route from using the same domain regardless of a path beeing specified.

To reproduce:
Route 1: test.example.com - passthrough - no issues
Route 2: test.example.com/ - edge - rejected >>> works as expected
Route 3: test.example.com - passthrough - rejected >>> works as expected
Route 4: test.example.com/test - edge - not useable, but no warning about the collision (router behaves correctly, check doesn't)

I only tested the behavior for HTTPS, I didn't test any behavior for HTTP or combinations of HTTP and HTTPS.

Thanks,
Keno

Comment 7 Hongan Li 2019-05-15 03:07:17 UTC
Passthrough termination does not support path, I cannot create passthrough route with a path when using both "oc create route" and "oc edit route" as below:  


# oc create route passthrough pass --service=service-secure --hostname=test.example.com --path=/path
Error: unknown flag: --path

# oc create route passthrough pass --service=service-secure --hostname=test.example.com 
route "pass" created
# oc get route
NAME      HOST/PORT          PATH      SERVICES         PORT      TERMINATION   WILDCARD
pass      test.example.com             service-secure   https     passthrough   None
# oc edit route pass
>>> try to add field "path" then save, but got below error:
# routes "pass" was not valid:
# * spec.path: Invalid value: "/path": passthrough termination does not support paths

Comment 8 Hongan Li 2019-05-15 03:32:46 UTC
created edge routes with/without path are also as expected

# oc create route edge edge --service=service-unsecure --hostname=test.example.com
route "edge" created

# oc create route edge edgewithpath --service=service-unsecure --hostname=test.example.com --path=/path
route "edgewithpath" created

# oc get route
NAME           HOST/PORT            PATH      SERVICES           PORT      TERMINATION   WILDCARD
edge           HostAlreadyClaimed             service-unsecure   http      edge          None
edgewithpath   test.example.com     /path     service-unsecure   http      edge          None
pass           test.example.com               service-secure     https     passthrough   None

note: edge route with hostname "test.example.com" is marked as "HostAlreadyClaimed" so should not be loaded to haproxy.config, see below:
# oc -n default rsh router-1-qx5bb
sh-4.2$ grep "hongli" haproxy.config             ### "hongli" is my test project name
backend be_edge_http:hongli:edgewithpath
backend be_tcp:hongli:pass

Comment 10 Hongan Li 2019-05-20 03:00:51 UTC
See the document of "Path based routes":
https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#path-based-routes

Routers should match routes based on the most specific path to the least. So when you curl “test.example.com/test” it will match edge route with path, if curl other URL like "test.example.com/" or "test.example.com/anything/" it will match your passthrough route. 

And I checked both route 1 and 3 and looks they are working as expected, here is the test result:
# oc get pod -o wide
NAME             READY     STATUS    RESTARTS   AGE       IP            NODE                         NOMINATED NODE
caddy-docker     1/1       Running   0          1h        10.129.0.18   preserve-hongli-311-node-1   <none>
caddy-docker-2   1/1       Running   0          35m       10.129.0.19   preserve-hongli-311-node-1   <none>

# oc get svc -o wide
NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE       SELECTOR
service-secure       ClusterIP   172.30.180.227   <none>        27443/TCP   1h        name=caddy-docker
service-unsecure-2   ClusterIP   172.30.17.255    <none>        27017/TCP   35m       name=caddy-docker-2

# oc get route
NAME           HOST/PORT            PATH      SERVICES             PORT      TERMINATION   WILDCARD
edge           HostAlreadyClaimed             service-unsecure-2   http      edge          None
edgewithpath   test.example.com     /path     service-unsecure-2   http      edge          None
pass           test.example.com               service-secure       https     passthrough   None

# curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/path/ -k
ocp-test https-8443

# curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/ -k
Hello-OpenShift-1 https-8443

# curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/test/ -k
Hello-OpenShift-Path-Test https-8443

Comment 11 Hongan Li 2019-05-20 03:17:58 UTC
In the testing, if you create "route 1" with re-encrypt route instead of the passthrough, you should get the same results.

If you still has "not useable" issue for any routes, please provide your test steps and related resources including "oc get pod -o yaml, oc get svc -o yaml, oc get route -o yaml", and all map files and haproxy.config in router pod.

sh-4.2$ ls haproxy.config 
haproxy.config
sh-4.2$ ls *.map
cert_config.map  os_edge_reencrypt_be.map  os_http_be.map  os_route_http_redirect.map  os_sni_passthrough.map  os_tcp_be.map  os_wildcard_domain.map

Thanks,

Comment 12 Keno Engelbracht 2019-06-11 18:05:00 UTC
Hi,
the "problem" is that the UI behavior doesn't represent the expected behavior. I just verified the problem again. 
For reproduction the order of creating the routes is important, please see below:

test1    test.example.com            nginx-example   web  passthrough   None | first route: works as expected: "exposed on router router 5 minutes ago"
test2    test.example.com  /test/    nginx-example   web  edge          None | second route: should throw an error, as domain is already claimed by route test1. Instead: "exposed on router router 4 minutes ago"
test3    HostAlreadyClaimed          nginx-example   web  edge          None | third route: throws an error, as domain is already claimed by route test1: "route test1 already exposes test.example.com and is older"

As all traffic is passed through to route test1 and the traffic can't be separated to go to any more specific routes (in this example: to test2).
Both route test1 and test2 claim the same path (the passthrough route basicly claims all paths for that domain). Route test2 MUST throw an error, even if it seems more specific, as specificity doesn't matter here, just age. 

The router itself works correctly as it services route test1 (it is the older route) but not service route test2 (the traffic for test2 can not be separated from the traffic for route test2).

Comment 14 Keno Engelbracht 2019-06-17 08:34:38 UTC
Hi Hongan,
I just used that single service to directly show the UI behavior. The router works correctly with multiple services (as I tested after our customer reported "issues" with one of the routes they created after the passthrough route).
No matter how this is set up routes other than test1 MUST NOT work. The encrypted traffic for test1 is routed to the correct pod via SNI, any path information is encrypted and can't be known to the router without the router doing a MitM on the connection, therefore the router can't possibly differenciate on the path if the first route is passthrough.

Comment 15 Hongan Li 2019-06-17 09:21:22 UTC
Hi Keno, thanks for your patience and explanation. I think finally I reproduced the issue you described. 

You're right. I re-test the steps in Comment 10 and find I mistook one of the test result below: 

############# it was from caddy-docker but not caddy-docker-2  #########
# curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/path/ -k
ocp-test https-8443
#############

Since `curl` results are from same backend pod, so that confirms all traffic is passed through to the passthrough route.

Comment 16 Hongan Li 2019-06-17 09:35:12 UTC
Hi @Dan, could you help take a look?

Since the original description of problem is confusing, you can skip it and focus Keno's comments (Comment 2, Comment 12).

Comment 20 Miciah Dashiel Butler Masters 2019-10-28 23:11:29 UTC
https://github.com/openshift/router/pull/57 should fix the problem.  I'll retarget this Bugzilla report to OCP 4.3, and we can clone the report for earlier versions if backports are requested.

Comment 22 Hongan Li 2019-10-31 08:50:59 UTC
verified with 4.3.0-0.nightly-2019-10-30-223128 and passed.

create passthrough route firstly, then try to create http, edge and reencrypt route with same host and a path, only http can work, others are showed as "HostAlreadyClaimed".

$ oc get route
NAME                 HOST/PORT                                                       PATH     SERVICES             PORT    TERMINATION   WILDCARD
reen                 HostAlreadyClaimed                                              /path/   service-secure-2     https   reencrypt     None
service-secure       example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com            service-secure       https   passthrough   None
service-unsecure-2   HostAlreadyClaimed                                              /test/   service-unsecure-2   http    edge          None
test2                example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com   /test/   service-unsecure-2   http                  None

Comment 23 Hongan Li 2019-10-31 09:03:57 UTC
I did more testing but found it still has issue if creating edge route with path firstly then creating passthrough route with same hostname.
Sorry I need reopen this for further investigation.


$ oc get route
NAME                 HOST/PORT                                                       PATH     SERVICES             PORT    TERMINATION   WILDCARD
service-secure       example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com            service-secure       https   passthrough   None
service-unsecure-2   example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com   /test/   service-unsecure-2   http    edge          None

Comment 25 Hongan Li 2019-11-05 06:58:22 UTC
verified with 4.3.0-0.nightly-2019-11-02-092336 and issue has been fixed.

Comment 27 errata-xmlrpc 2020-01-23 11:03:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.