Description of problem: - When creating a wildcard ingress the route is never created due to a mismatch in the host format in both resources. - In k8s ingress the hostname is configured as '*.domain.tld' (see [1]) but that's not a valid hostname for a route, and the conversion code copies it literally as seen in [2], failing in the creation. Version-Release number of selected component (if applicable): - 4.6.4 How reproducible: - Always Additional info: - The error can be seen in the controller after setting Debug logLevel: I1202 08:42:11.429894 1 ingress.go:342] Error syncing {dev route-wildcard-2}: Route.route.openshift.io "route-wildcard-2-zkcdj" is invalid: spec.host: Invalid value: "*.dev.apps.sgarciam-02807641.emeashift.support": host must conform to DNS 952 subdomain conventions - Valid k8s ingress: kind: Ingress apiVersion: networking.k8s.io/v1beta1 metadata: name: ingress-1 spec: rules: - host: '*.ingress-1.domain.tld' http: paths: - path: / pathType: Prefix backend: serviceName: ingress-1-service servicePort: 80 - Expected route: kind: Route apiVersion: route.openshift.io/v1 metadata: name: route-1 spec: host: ingress-1.domain.tld to: kind: Service name: ingress-1-service port: targetPort: 80-tcp wildcardPolicy: Subdomain References: [1] https://kubernetes.io/docs/concepts/services-networking/ingress/#hostname-wildcards [2] https://github.com/openshift/openshift-controller-manager/blob/master/pkg/route/ingress/ingress.go#L555
Working on this now https://github.com/openshift/openshift-controller-manager/pull/169 It is still Work in Progress.
@misalunk We are having this same issue in 4.6.12 and want to make sure that it is backported to 4.6.x.
The fix is working fine. Unit test have passed. Passing to QE. Manual testing -> oc new-project wildcard oc new-app django-psql-example [miheer@miheer openshift-controller-manager]$ cat ingress-wildcard.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: ingress1 annotations: haproxy.router.openshift.io/timeout: 5m namespace: wildcard spec: rules: - host: "*.testing.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com" http: paths: - path: / backend: serviceName: django-psql-example servicePort: 8080 [miheer@miheer openshift-controller-manager]$ [miheer@miheer openshift-controller-manager]$ oc create -f ingress-wildcard.yaml W0217 14:05:31.500559 50971 warnings.go:67] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress ingress.networking.k8s.io/ingress1 created [miheer@miheer openshift-controller-manager]$ oc get routes --all-namespaces NAMESPACE NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD openshift-authentication oauth-openshift oauth-openshift.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com oauth-openshift 6443 passthrough/Redirect None openshift-console console console-openshift-console.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com console https reencrypt/Redirect None openshift-console downloads downloads-openshift-console.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com downloads http edge/Redirect None openshift-ingress-canary canary canary-openshift-ingress-canary.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com ingress-canary 8080 None openshift-monitoring alertmanager-main alertmanager-main-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com alertmanager-main web reencrypt/Redirect None openshift-monitoring grafana grafana-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com grafana https reencrypt/Redirect None openshift-monitoring prometheus-k8s prometheus-k8s-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com prometheus-k8s web reencrypt/Redirect None openshift-monitoring thanos-querier thanos-querier-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com thanos-querier web reencrypt/Redirect None wildcard django-psql-example django-psql-example-wildcard.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com django-psql-example <all> None wildcard ingress1-xmsqh wildcard.testing.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com / django-psql-example web Subdomain [miheer@miheer openshift-controller-manager]$ [miheer@miheer openshift-controller-manager]$ [miheer@miheer openshift-controller-manager]$ [miheer@miheer openshift-controller-manager]$ Modify ingress. I just changed "*.testing" in spec.rules.host to "*.testing1" [miheer@miheer openshift-controller-manager]$ [miheer@miheer openshift-controller-manager]$ oc edit ing ingress1 ingress.networking.k8s.io/ingress1 edited [miheer@miheer openshift-controller-manager]$ oc get routes --all-namespaces NAMESPACE NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD openshift-authentication oauth-openshift oauth-openshift.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com oauth-openshift 6443 passthrough/Redirect None openshift-console console console-openshift-console.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com console https reencrypt/Redirect None openshift-console downloads downloads-openshift-console.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com downloads http edge/Redirect None openshift-ingress-canary canary canary-openshift-ingress-canary.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com ingress-canary 8080 None openshift-monitoring alertmanager-main alertmanager-main-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com alertmanager-main web reencrypt/Redirect None openshift-monitoring grafana grafana-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com grafana https reencrypt/Redirect None openshift-monitoring prometheus-k8s prometheus-k8s-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com prometheus-k8s web reencrypt/Redirect None openshift-monitoring thanos-querier thanos-querier-openshift-monitoring.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com thanos-querier web reencrypt/Redirect None wildcard django-psql-example django-psql-example-wildcard.apps.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com django-psql-example <all> None wildcard ingress1-gt9hw wildcard.testing1.ci-ln-03c9nrb-d5d6b.origin-ci-int-aws.dev.rhcloud.com / django-psql-example web Subdomain [miheer@miheer openshift-controller-manager]$
(In reply to Dana from comment #4) > @misalunk We are having this same issue in 4.6.12 and want to > make sure that it is backported to 4.6.x. OK we will see what can be done as I will need to discuss with our team on that.
You need run this as pre-requisite for allowing wildcard routes. [miheer@miheer openshift-controller-manager]$ oc patch -n openshift-ingress-operator ingresscontroller/default --patch '{"spec":{"routeAdmission":{"wildcardPolicy": "WildcardsAllowed"}}}' --type=merge ingresscontroller.operator.openshift.io/default patched
Verified in "4.8.0-0.ci.test-2021-02-17-044628-ci-ln-1ldbxqb" release version. With this payload the ingress resource could be seen getting translated correctly to the route resource with the required wildcard mapping: --------- $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.8.0-0.ci.test-2021-02-17-044628-ci-ln-1ldbxqb True False 51m Cluster version is 4.8.0-0.ci.test-2021-02-17-044628-ci-ln-1ldbxqb $ oc patch -n openshift-ingress-operator ingresscontroller/internalapps --patch '{"spec":{"routeAdmission":{"wildcardPolicy": "WildcardsAllowed"}}}' --type=merge ingresscontroller.operator.openshift.io/default patched $ cat test-ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: unsecure-route spec: rules: - host: '*.test1.internalapps.ci-ln-1ldbxqb-d5d6b.origin-ci-int-aws.dev.rhcloud.com' http: paths: - path: "/" pathType: "Prefix" backend: service: name: service-unsecure port: number: 27017 $ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD unsecure-route-2f4w4 wildcard.test1.internalapps.ci-ln-1ldbxqb-d5d6b.origin-ci-int-aws.dev.rhcloud.com ... 1 more / service-unsecure http Subdomain $ oc get ingress NAME CLASS HOSTS ADDRESS PORTS AGE unsecure-route <none> *.test1.internalapps.ci-ln-1ldbxqb-d5d6b.origin-ci-int-aws.dev.rhcloud.com apps.ci-ln-1ldbxqb-d5d6b.origin-ci-int-aws.dev.rhcloud.com,internalapps.ci-ln-1ldbxqb-d5d6b.origin-ci-int-aws.dev.rhcloud.com 80 13m ---------
Any news on if this will be backported to 4.6 and 4.7?
We discussed this BZ and decided that we will backport the fix to 4.8, 4.7, and 4.6. However, there is a small risk in backporting this change. Suppose the following: * A user has an ingress with a wildcard host name. * The ingress is exposed using a third-party ingress controller. * The user does *not* want the ingress exposed by OpenShift router (for example, maybe OpenShift router is exposed on the Internet, and the third-party ingress controller is internal only; or maybe having a status entry from OpenShift router on the ingress would confuse the third-party ingress controller, as in bug 1935808). * The router is configured to allow wildcard routes (as described in comment 8). * The user upgrades to a version of OpenShift with this backported fix. In this sceneario, the ingress is now exposed by the router when the user only wants it exposed by the third-party ingress controller. This is an admittedly extremely contrived situation, but sometimes users do unexpected things or have different expectations, and exposing an ingress when the user doesn't expect it could be problematic. Therefore we need to document this new behavior very explicitly in a release note to reduce the risk that users will be caught unawares. Miheer, could you write a release note and proceed with the backport?
Verified in "4.9.0-0.nightly-2021-08-14-065522" release version, this time. With this payload the ingress resource could be seen getting translated correctly to the route resource with the required wildcard mapping: --------- oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.9.0-0.nightly-2021-08-14-065522 True False 173m Cluster version is 4.9.0-0.nightly-2021-08-14-065522 $ oc patch -n openshift-ingress-operator ingresscontroller/internalapps --patch '{"spec":{"routeAdmission":{"wildcardPolicy": "WildcardsAllowed"}}}' --type=merge ingresscontroller.operator.openshift.io/default patched $ cat test-ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: unsecure-route spec: rules: - host: '*.test1.internalapps.aiyengar4916.qe.devcluster.openshift.com' http: paths: - path: "/" pathType: "Prefix" backend: service: name: service-unsecure port: number: 27017 oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD unsecure-route-ghq74 wildcard.test1.internalapps.aiyengar4916.qe.devcluster.openshift.com ... 1 more / service-unsecure http Subdomain oc get ingress NAME CLASS HOSTS ADDRESS PORTS AGE unsecure-route <none> *.test1.internalapps.aiyengar4916.qe.devcluster.openshift.com 80 38s curl -I http://wildcard.test1.internalapps.aiyengar4916.qe.devcluster.openshift.comserver: nginx/1.18.0 date: Mon, 16 Aug 2021 06:25:03 GMT content-type: text/html content-length: 46 last-modified: Mon, 16 Aug 2021 06:14:23 GMT etag: "611a023f-2e" accept-ranges: bytes set-cookie: 491bb4f494b480a2be42674d291d426f=a37c3b8df559f554ce53bfe738097aab; path=/; HttpOnly cache-control: private ---------
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759