Description of problem: When configuring ingress resource with "route.openshift.io/termination" annotation set to "passthrough" via the "networking.k8s.io/v1" version. Corresponding route fails to get created. The issue specifically appears when trying to define "Passthrough" in the object while it works fine for "edge" or "reen" type. The annotation works perfectly when using the older "extensions/v1beta1" API version and the routes get created and remain accessible. Version-Release number of selected component (if applicable): * cluster version ----- NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.6.0-fc.5 True False 5h18m Cluster version is 4.6.0-fc.5 ---- How reproducible: Frequently. Steps to Reproduce: 1. Deploy project/service/pods 2. Define an ingress object via yaml file with "apiVersion: networking.k8s.io/v1" ------ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-secure annotations: route.openshift.io/termination: "passthrough" spec: rules: - host: service-secure-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com http: paths: - path: "/" backend: service: name: service-secure port: number: 27443 pathType: "Prefix" ------ 3. create the object 4. Verify the "oc get ingresses" and "oc get route" in the project Actual results: The file gets read and the ingress object gets created: ---- $ oc create -f ../essential-docs/test-files/ingress-passthrough-k8s.yaml ingress.networking.k8s.io/ingress-secure created ---- But the corresponding route object does not get created: ---- $ oc get ingresses NAME CLASS HOSTS ADDRESS PORTS AGE ingress-secure <none> service-secure-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com 80 78m $ oc get route -o wide NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD ---- Expected results: The creation of ingress object with the latest "networking.k8s" API should result in the route object being created Additional info: This only seem to occur for "passthrough" type annotations. For other types like "Edge" or "Reencrypt" the corresponding route gets created and remains reachable as expected. Reference file used for edge annotated ingress object as an example: ---- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-edge annotations: route.openshift.io/termination: "edge" spec: tls: - hosts: - service-unsecure-edge-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com secretName: edge-secret rules: - host: service-unsecure-edge-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com http: paths: - path: "/" pathType: "Prefix" backend: service: name: service-unsecure port: number: 27017 $ oc create -f ../essential-docs/test-files/ingress-edge-k8s.yaml ingress.networking.k8s.io/ingress-edge created $ oc get ingress NAME CLASS HOSTS ADDRESS PORTS AGE ingress-edge <none> service-unsecure-edge-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com 80, 443 6s $ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD ingress-edge-4bpds service-unsecure-edge-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com ... 1 more / service-unsecure http edge/Redirect None $ curl https://service-unsecure-edge-test1.internalapps.oc465-hackthon-1109.qe.devcluster.openshift.com/ -k Hello-OpenShift-1 http-8080 ----
Tagging as 4.7 and will consider a backport once the problem is understood.
I'll investigate the issue in the upcoming sprint.
In OpenShift 4.6.6 this behavior (no routes for passthrough) appears to be present in both the v1beta1 and v1 Network APIs. I've also noticed that when defining an Ingress object in v1, it actually gets created as v1beta1 e.g., I create the object using v1 data structures through the API and then when I ask for it back via `oc get ingress <blah> -o yaml` I see `apiVersion: extensions/v1beta1`.
Working on this. The controller code from the unit test cases seems to be creating a route properly. https://github.com/openshift/openshift-controller-manager/pull/187/commits/cb9f9387191f2ac8fc3a42297db9c669dee071e9 I will have to investigate this more deeper.
Fix available in https://github.com/openshift/openshift-controller-manager/pull/187/files The issue was -> time="2021-05-27T12:12:50Z" level=info msg="errs is [Route.route.openshift.io \"ingress1-6sdgd\" is invalid: spec.path: Invalid value: \"/\": passthrough termination does not support paths]" [miheer@localhost openshift-controller-manager]$ Tests are working fine. [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ cat test-ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress1 namespace: test-ingress annotations: route.openshift.io/termination: passthrough spec: rules: - host: ingress-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support http: paths: - path: / pathType: Prefix backend: service: name: django-psql-example port: number: 8080 [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ oc get ing NAME CLASS HOSTS ADDRESS PORTS AGE ingress1 <none> ingress-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support apps.misalunktls13bug.apacshift.support 80 15m ingress6 <none> ingress6-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support apps.misalunktls13bug.apacshift.support 80 4m33s [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ oc delete ing ingress1 ingress.networking.k8s.io "ingress1" deleted [miheer@localhost openshift-controller-manager]$ oc get routes NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD django-psql-example django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example <all> None ingress6-jqqrz ingress6-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example web passthrough/Redirect None [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ oc create -f test-ingress.yaml ingress.networking.k8s.io/ingress1 created [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ oc get ing NAME CLASS HOSTS ADDRESS PORTS AGE ingress1 <none> ingress-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support apps.misalunktls13bug.apacshift.support 80 5s ingress6 <none> ingress6-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support apps.misalunktls13bug.apacshift.support 80 5m15s [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ oc get routes NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD django-psql-example django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example <all> None ingress1-zrxlr ingress-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example web passthrough/Redirect None ingress6-jqqrz ingress6-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example web passthrough/Redirect None [miheer@localhost openshift-controller-manager]$
Creating a passthrough route using pathType: ImplementationSpecific works apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: route.openshift.io/termination: passthrough name: example namespace: openshift-ingress spec: rules: - host: example.com http: paths: - backend: service: name: router-internal-default port: number: 443 pathType: ImplementationSpecific % oc -n openshift-ingress get ingresses,routes NAME CLASS HOSTS ADDRESS PORTS AGE ingress.networking.k8s.io/example <none> example.com apps.ci-ln-fxc87nk-f76d1.origin-ci-int-gce.dev.openshift.com 80 59s NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD route.route.openshift.io/example-k7mlv example.com router-internal-default https passthrough/Redirect None % So fixing the code is not required. @aiyengar can you please check this ? We will be just making a note of this.
Hi Arvind, Keep path: '' [miheer@localhost openshift-controller-manager]$ cat test-ingress7.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress7 namespace: test-ingress annotations: route.openshift.io/termination: passthrough spec: rules: - host: ingress7-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support http: paths: - path: '' pathType: ImplementationSpecific backend: service: name: django-psql-example port: number: 8080 [miheer@localhost openshift-controller-manager]$ [miheer@localhost openshift-controller-manager]$ oc create -f test-ingress7.yaml ingress.networking.k8s.io/ingress7 created [miheer@localhost openshift-controller-manager]$ oc get routes NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD django-psql-example django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example <all> None ingress1-2ht78 ingress-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example web passthrough/Redirect None ingress6-jqqrz ingress6-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example web passthrough/Redirect None ingress7-6fdqm ingress7-django-psql-example-test-ingress.apps.misalunktls13bug.apacshift.support django-psql-example web passthrough/Redirect None Thanks and regards, Miheer Salunke
Hi Miheer, Thanks for the clarification, retesting using the mentioned method in the previous comment, it is observed that the corresponding routes are getting created for the ingress resource: ------ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.8.0-0.nightly-2021-06-13-101614 True False 23h Cluster version is 4.8.0-0.nightly-2021-06-13-101614 ingress-passthrough-k8s-1878685.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-passth annotations: route.openshift.io/termination: passthrough spec: rules: - host: django-psql-example2-test4.apps.aiyengar4814.qe.devcluster.openshift.com http: paths: - path: '' pathType: ImplementationSpecific backend: service: name: django-psql-example port: number: 8080 oc create -f ingress-passthrough-k8s-1878685.yaml ingress.networking.k8s.io/ingress-passth created oc get ingress NAME CLASS HOSTS ADDRESS PORTS AGE ingress-passth <none> django-psql-example2-test4.apps.aiyengar4814.qe.devcluster.openshift.com router-default.apps.aiyengar4814.qe.devcluster.openshift.com 80 32s oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD ingress-passth-tnm4r django-psql-example2-test4.apps.aiyengar4814.qe.devcluster.openshift.com django-psql-example web passthrough/Redirect None ------
Marking as closed. Link to live doc here: https://docs.openshift.com/container-platform/4.7/networking/routes/route-configuration.html#nw-ingress-creating-a-route-via-an-ingress_route-configuration