Bug 1653633

Summary: [Next_gen_installer] user cannot access the routes
Product: OpenShift Container Platform Reporter: Hongan Li <hongli>
Component: RoutingAssignee: Dan Mace <dmace>
Status: CLOSED WORKSFORME QA Contact: Hongan Li <hongli>
Severity: high Docs Contact:
Priority: high    
Version: 4.1.0CC: aos-bugs, wsun, xxia
Target Milestone: ---Keywords: TestBlocker
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-07 06:04:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Hongan Li 2018-11-27 10:24:46 UTC
Description of problem:
The user cannot access the routes

Version-Release number of selected component (if applicable):
# bin/openshift-install version
bin/openshift-install v0.4.0-10-ge15d801ad69481da18d409bec5fa1c7bd7998f3a
Terraform v0.11.8

$ oc version
oc v4.0.0-0.63.0
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://qe-test-api.devcluster.openshift.com:6443
kubernetes v1.11.0+d4cacc0

How reproducible:

Steps to Reproduce:
1. Launch the cluster via aws provider.
2. create your project, pod, svc and route.
oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/caddy-docker.json
oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/edge/service_unsecure.json
oc expose svc service-unsecure

3. check the route
$ oc get route -n hongli
NAME               HOST/PORT                                                  PATH      SERVICES           PORT      TERMINATION   WILDCARD
service-unsecure   service-unsecure-hongli.router.default.svc.cluster.local             service-unsecure   http                    None

Actual results:
The user cannot access the route.

Expected results:
The route should be accessed from both outside cluster and inside cluster.

Additional info:
1. the route sub-domain cannot be configured, and no DNS for the domain.
2. checked the default router service and found it use LB and node port, but the node port didn't respond to any request.
3. resolve the lb to IP then use lb's IP as resolved IP when curling and get the response, seems it is a workaround but not suitable for regression testing.  

$ oc get svc -n openshift-ingress
NAME             TYPE           CLUSTER-IP    EXTERNAL-IP                                                              PORT(S)                      AGE
router-default   LoadBalancer   ac9e63418f1e811e88beb0ae40ba6a02-984794043.us-east-1.elb.amazonaws.com   80:30944/TCP,443:32470/TCP   8h

$ nslookup ac9e63418f1e811e88beb0ae40ba6a02-984794043.us-east-1.elb.amazonaws.com

Non-authoritative answer:
Name:	ac9e63418f1e811e88beb0ae40ba6a02-984794043.us-east-1.elb.amazonaws.com
Name:	ac9e63418f1e811e88beb0ae40ba6a02-984794043.us-east-1.elb.amazonaws.com
Name:	ac9e63418f1e811e88beb0ae40ba6a02-984794043.us-east-1.elb.amazonaws.com

$ curl --resolve service-unsecure-hongli.router.default.svc.cluster.local:80: http://service-unsecure-hongli.router.default.svc.cluster.local
Hello-OpenShift-1 http-8080

Comment 1 Hongan Li 2018-12-07 05:57:26 UTC
routing subdomain issue was fixed by PR: https://github.com/openshift/cluster-openshift-apiserver-operator/pull/63

Comment 2 Hongan Li 2018-12-07 06:04:18 UTC
Checked with version v0.5.0-master-14-g8c504c011e5ce8c28c9fb383e5861e1c70353c82 on AWS, the routes have resolvable subdomain and are reachable now.

$ oc get route -n hongli
NAME               HOST/PORT                                                               PATH      SERVICES           PORT      TERMINATION   WILDCARD
service-unsecure   service-unsecure-hongli.apps.hongli.origin-ci-int-aws.dev.rhcloud.com             service-unsecure   http                    None

$ oc get ingresses.config.openshift.io/cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Ingress
  creationTimestamp: 2018-12-06T01:16:26Z
  generation: 1
  name: cluster
  resourceVersion: "236"
  selfLink: /apis/config.openshift.io/v1/ingresses/cluster
  uid: 8da1b8d4-f8f4-11e8-a6ff-06b256c6bf30
  domain: apps.hongli.origin-ci-int-aws.dev.rhcloud.com
status: {}