Bug 1558202 - Routes created with wrong dns suffix when using router shards
Summary: Routes created with wrong dns suffix when using router shards
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Routing
Version: 3.7.0
Hardware: All
OS: Linux
Target Milestone: ---
: 3.10.0
Assignee: Ram Ranganathan
QA Contact: zhaozhanqi
Depends On:
TreeView+ depends on / blocked
Reported: 2018-03-19 19:59 UTC by Anshul Verma
Modified: 2018-07-23 13:12 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-05-18 15:06:07 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Origin (Github) 19418 None None None 2018-04-26 18:50:45 UTC

Description Anshul Verma 2018-03-19 19:59:01 UTC
Description of problem:

Context -
${name}-${namespace}.internal.example.com - is default RouterConfig subdomain set in master-config, ROUTE_LABELS="shard notin (public, trusted)".
${name}-${namespace}.public.example.com. -  public router shard is created with ROUTE_LABELS="shard=public".

While using router sharding -
If ROUTER_OVERRIDE_HOSTNAME="true" is set on the routers, everything work as expected.

But if ROUTER_OVERRIDE_HOSTNAME="false" is on the routers, all "empty" routes are admitted as
${name}-${namespace}.internal.example.com on the routers.
Setting a route label to "shard=public" is creating the route on the public router but with a hostname of ${name}-${namespace}.internal.example.com, therefore not respecting the ROUTER_SUBDOMAIN, thus making it impossible to direct traffic to the route.

Expected behaviour should be that the route controller respects the ROUTER_SUBDOMAIN of the router when admitting empty routes claimed by a router shard when ROUTER_OVERRIDE_HOSTNAME="false" is set.

An upstream issue has already been created -
And someone has already posted his code with introducing --ignore-domain option, but no PR has be filed yet for the same.

Version-Release number of selected component (if applicable):

Comment 2 Ram Ranganathan 2018-04-12 19:54:47 UTC
There's two router options that control blacklisting and whitelisting route domains.
Documentation at: https://docs.openshift.org/latest/architecture/networking/routes.html#architecture-core-concepts-routes-deny-allow

So doing something like:

oc set env dc/public-router-dc ROUTER_DENIED_DOMAINS="internal.example.com"

should work on the public and/or trusted router.

Its doing the same check as ignore domains - just on a set of domains. In a similar vein, there's also domain whitelist (allowed domains) that one can use if you know the list of domains on routes that you want a specific router to serve.

Does that work for you? Thanks.

Comment 3 Ram Ranganathan 2018-04-18 23:37:26 UTC
Okay, there was correspondence with the customer on the github issue and that clarified what they were asking for. Or my misunderstanding of it anyway!

There's an in-progress PR for this: https://github.com/openshift/origin/pull/19418

if that works, will add some tests to it and push it.

Comment 6 zhaozhanqi 2018-05-18 06:55:04 UTC
found this bug still can be reproduced on v3.10.0-0.47.0


1.  Create two routers with different ROUTER_SUBDOMAIN and ROUTE_LABELS

  oc env dc router ROUTER_SUBDOMAIN='${name}-${namespace}.inter.example.com' ROUTE_LABELS="shard=inter"
  oc env dc r2 ROUTER_SUBDOMAIN='${name}-${namespace}.public.example.com' ROUTE_LABELS="shard=public

2. update router r2 with 

 oc env dc r2 ROUTER_OVERRIDE_HOSTNAME=false ROUTER_OVERRIDE_DOMAINS='public.example.com'

3. Create one service with userA

  oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/header-test/insecure-service.json

4. Create route with label shard=public
   oc expose svc header-test-insecure -l shard=public --name=zhao2

5. Check the route
  oc get route zhao2
NAME      HOST/PORT                               PATH      SERVICES               PORT      TERMINATION   WILDCARD
zhao2     zhao2-z1.apps.0517-llp.qe.rhcloud.com             header-test-insecure   http                    None

6.  oc describe route zhao2
Name:			zhao2
Namespace:		z1
Created:		15 minutes ago
Labels:			shard=public
Annotations:		openshift.io/host.generated=true
Requested Host:		zhao2-z1.apps.0517-llp.qe.rhcloud.com
			  exposed on router r2 9 minutes ago
Path:			<none>
TLS Termination:	<none>
Insecure Policy:	<none>
Endpoint Port:		http

Service:	header-test-insecure
Weight:		100 (100%)
Endpoints:	<none>

FYI: the subdomin of in master-config.yaml is 'apps.0517-llp.qe.rhcloud.com'

Comment 7 Ben Bennett 2018-05-18 15:06:07 UTC
I'm not sure what you expect to have happened.  That all looks correct to me.

By setting ROUTER_OVERRIDE_HOSTNAME=false and ROUTER_OVERRIDE_DOMAINS='public.example.com' you are requesting that the router not override all hostnames, only the ones in public.example.com.

BUT your route hostname is zhao2-z1.apps.0517-llp.qe.rhcloud.com.  It needs to be in the domain public.example.com for it to get overriden.

Note You need to log in before you can comment on or make changes to this bug.