Description of problem: Passthrough routes does not support path based routing. Why this route can be created? It should throw an error. But actual result is Route gets created and that is unusable. Replicte 1st scenario: Create a pass-through route with --path variable and it gets created without an error. 2nd Scenario: Create a simple pass-through route. Create one more pass-through route with same hostname as above and give a path to the route. It also gets created. Actual results: it is processed same as other routes. Expected results: In both cases, It should throw an error, While creating pass-through route it should not check for hostname+path combination, and should throw an error stating not supported.
Hello Team, I would like to add one more point which I missed in description for the scenario 2, In which we are creating a new route with hostname already being used by a passthrough route, when you also specify a path for that new route, it gets created without a error. Actual Results: The duplicate domain detection does not recognize the domain part of this new route when specified a path to the new route. Expected: It should throw an error and the duplicate domain detection should not treat the passthrough route as other routes. Thanks.
Hi, to amend the initial description: The problem occurs when edge terminated route that specifies a path is created after a passthrough route using the same domain. Only then it is not detected by the domain collision prevention, because the check only looks for routes with the same "path" parameter, which the algorithm considers to be "/" for a passthrough route instead of a wildcard which would would prevent any additional route from using the same domain regardless of a path beeing specified. To reproduce: Route 1: test.example.com - passthrough - no issues Route 2: test.example.com/ - edge - rejected >>> works as expected Route 3: test.example.com - passthrough - rejected >>> works as expected Route 4: test.example.com/test - edge - not useable, but no warning about the collision (router behaves correctly, check doesn't) I only tested the behavior for HTTPS, I didn't test any behavior for HTTP or combinations of HTTP and HTTPS. Thanks, Keno
Passthrough termination does not support path, I cannot create passthrough route with a path when using both "oc create route" and "oc edit route" as below: # oc create route passthrough pass --service=service-secure --hostname=test.example.com --path=/path Error: unknown flag: --path # oc create route passthrough pass --service=service-secure --hostname=test.example.com route "pass" created # oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD pass test.example.com service-secure https passthrough None # oc edit route pass >>> try to add field "path" then save, but got below error: # routes "pass" was not valid: # * spec.path: Invalid value: "/path": passthrough termination does not support paths
created edge routes with/without path are also as expected # oc create route edge edge --service=service-unsecure --hostname=test.example.com route "edge" created # oc create route edge edgewithpath --service=service-unsecure --hostname=test.example.com --path=/path route "edgewithpath" created # oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD edge HostAlreadyClaimed service-unsecure http edge None edgewithpath test.example.com /path service-unsecure http edge None pass test.example.com service-secure https passthrough None note: edge route with hostname "test.example.com" is marked as "HostAlreadyClaimed" so should not be loaded to haproxy.config, see below: # oc -n default rsh router-1-qx5bb sh-4.2$ grep "hongli" haproxy.config ### "hongli" is my test project name backend be_edge_http:hongli:edgewithpath backend be_tcp:hongli:pass
See the document of "Path based routes": https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#path-based-routes Routers should match routes based on the most specific path to the least. So when you curl “test.example.com/test” it will match edge route with path, if curl other URL like "test.example.com/" or "test.example.com/anything/" it will match your passthrough route. And I checked both route 1 and 3 and looks they are working as expected, here is the test result: # oc get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE caddy-docker 1/1 Running 0 1h 10.129.0.18 preserve-hongli-311-node-1 <none> caddy-docker-2 1/1 Running 0 35m 10.129.0.19 preserve-hongli-311-node-1 <none> # oc get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service-secure ClusterIP 172.30.180.227 <none> 27443/TCP 1h name=caddy-docker service-unsecure-2 ClusterIP 172.30.17.255 <none> 27017/TCP 35m name=caddy-docker-2 # oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD edge HostAlreadyClaimed service-unsecure-2 http edge None edgewithpath test.example.com /path service-unsecure-2 http edge None pass test.example.com service-secure https passthrough None # curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/path/ -k ocp-test https-8443 # curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/ -k Hello-OpenShift-1 https-8443 # curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/test/ -k Hello-OpenShift-Path-Test https-8443
In the testing, if you create "route 1" with re-encrypt route instead of the passthrough, you should get the same results. If you still has "not useable" issue for any routes, please provide your test steps and related resources including "oc get pod -o yaml, oc get svc -o yaml, oc get route -o yaml", and all map files and haproxy.config in router pod. sh-4.2$ ls haproxy.config haproxy.config sh-4.2$ ls *.map cert_config.map os_edge_reencrypt_be.map os_http_be.map os_route_http_redirect.map os_sni_passthrough.map os_tcp_be.map os_wildcard_domain.map Thanks,
Hi, the "problem" is that the UI behavior doesn't represent the expected behavior. I just verified the problem again. For reproduction the order of creating the routes is important, please see below: test1 test.example.com nginx-example web passthrough None | first route: works as expected: "exposed on router router 5 minutes ago" test2 test.example.com /test/ nginx-example web edge None | second route: should throw an error, as domain is already claimed by route test1. Instead: "exposed on router router 4 minutes ago" test3 HostAlreadyClaimed nginx-example web edge None | third route: throws an error, as domain is already claimed by route test1: "route test1 already exposes test.example.com and is older" As all traffic is passed through to route test1 and the traffic can't be separated to go to any more specific routes (in this example: to test2). Both route test1 and test2 claim the same path (the passthrough route basicly claims all paths for that domain). Route test2 MUST throw an error, even if it seems more specific, as specificity doesn't matter here, just age. The router itself works correctly as it services route test1 (it is the older route) but not service route test2 (the traffic for test2 can not be separated from the traffic for route test2).
Hi Hongan, I just used that single service to directly show the UI behavior. The router works correctly with multiple services (as I tested after our customer reported "issues" with one of the routes they created after the passthrough route). No matter how this is set up routes other than test1 MUST NOT work. The encrypted traffic for test1 is routed to the correct pod via SNI, any path information is encrypted and can't be known to the router without the router doing a MitM on the connection, therefore the router can't possibly differenciate on the path if the first route is passthrough.
Hi Keno, thanks for your patience and explanation. I think finally I reproduced the issue you described. You're right. I re-test the steps in Comment 10 and find I mistook one of the test result below: ############# it was from caddy-docker but not caddy-docker-2 ######### # curl --resolve test.example.com:443:10.0.76.211 https://test.example.com/path/ -k ocp-test https-8443 ############# Since `curl` results are from same backend pod, so that confirms all traffic is passed through to the passthrough route.
Hi @Dan, could you help take a look? Since the original description of problem is confusing, you can skip it and focus Keno's comments (Comment 2, Comment 12).
https://github.com/openshift/router/pull/57 should fix the problem. I'll retarget this Bugzilla report to OCP 4.3, and we can clone the report for earlier versions if backports are requested.
verified with 4.3.0-0.nightly-2019-10-30-223128 and passed. create passthrough route firstly, then try to create http, edge and reencrypt route with same host and a path, only http can work, others are showed as "HostAlreadyClaimed". $ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD reen HostAlreadyClaimed /path/ service-secure-2 https reencrypt None service-secure example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com service-secure https passthrough None service-unsecure-2 HostAlreadyClaimed /test/ service-unsecure-2 http edge None test2 example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com /test/ service-unsecure-2 http None
I did more testing but found it still has issue if creating edge route with path firstly then creating passthrough route with same hostname. Sorry I need reopen this for further investigation. $ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD service-secure example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com service-secure https passthrough None service-unsecure-2 example-hongli.apps.hongli-aws128.qe.devcluster.openshift.com /test/ service-unsecure-2 http edge None
verified with 4.3.0-0.nightly-2019-11-02-092336 and issue has been fixed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days