Hide Forgot
Description of problem: Create namespace router shards env, in p1 create p1-router and one route, in p2 create one reencrypte route, in p3 create one passthrough route, all p1,p2 and p3 have same namespace labels, passthrough route infor is wrong under /var/lib/haproxy/conf/os_sni_passthrough.map Version-Release number of selected component (if applicable): [root@dhcp-41-211 ~]# oc version oc v3.3.0.32 kubernetes v1.3.0+52492b4 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://dhcp-41-211.bos.redhat.com:8443 openshift v3.3.0.32 kubernetes v1.3.0+52492b4 How reproducible: Reproducible by following below steps Steps to Reproduce: [root@dhcp-41-211 ~]# oc new-project p1 Now using project "p1" on server "https://dhcp-41-211.bos.redhat.com:8443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git to build a new example application in Ruby. [root@dhcp-41-211 ~]# oc label namespace p1 "router=p2" namespace "p1" labeled [root@dhcp-41-211 ~]# oadm policy add-scc-to-user privileged -z user1 [root@dhcp-41-211 ~]# oadm router p1-router --replicas=0 --service-account=user1 -n p1 --host-network=true info: password for stats user admin has been set to phf1Xni5RD --> Creating router p1-router ... serviceaccount "user1" created error: rolebinding "router-p1-router-role" already exists deploymentconfig "p1-router" created service "p1-router" created --> Failed [root@dhcp-41-211 ~]# oadm policy add-cluster-role-to-user cluster-reader system:serviceaccount:p1:user1 [root@dhcp-41-211 ~]# oc env dc/p1-router NAMESPACE_LABELS="router=p2" deploymentconfig "p1-router" updated [root@dhcp-41-211 ~]# oc scale dc/p1-router --replicas=1 deploymentconfig "p1-router" scaled [root@dhcp-41-211 ~]# oc create -f https://raw.githubusercontent.com/weliang1/Openshift_Networking/master/OSE3.3/hello-openshift-twopods.json route "hello-route" created service "hello-service" created pod "hello-pod-1" created pod "hello-pod-2" created [root@dhcp-41-211 ~]# [root@dhcp-41-211 ~]# oc new-project p2 Now using project "p2" on server "https://dhcp-41-211.bos.redhat.com:8443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git to build a new example application in Ruby. [root@dhcp-41-211 ~]# oc label namespace p2 "router=p2" namespace "p2" labeled [root@dhcp-41-211 ~]# wget https://raw.githubusercontent.com/weliang1/Openshift_Networking/master/route.pem --2016-09-30 10:17:17-- https://raw.githubusercontent.com/weliang1/Openshift_Networking/master/route.pem Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.56.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.56.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1480 (1.4K) [text/plain] Saving to: ‘route.pem.7’ 100%[===================================================================================================================================>] 1,480 --.-K/s in 0s 2016-09-30 10:17:17 (476 MB/s) - ‘route.pem.7’ saved [1480/1480] [root@dhcp-41-211 ~]# oc create -f https://raw.githubusercontent.com/weliang1/Openshift_Networking/master/reencrypt-route.json route "route-reen" created service "service-secure" created pod "caddy-docker-1" created pod "caddy-docker-2" created [root@dhcp-41-211 ~]# [root@dhcp-41-211 ~]# [root@dhcp-41-211 ~]# oc new-project p3 Now using project "p3" on server "https://dhcp-41-211.bos.redhat.com:8443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git to build a new example application in Ruby. [root@dhcp-41-211 ~]# oc label namespace p3 "router=p2" namespace "p3" labeled [root@dhcp-41-211 ~]# oc create -f https://raw.githubusercontent.com/weliang1/Openshift_Networking/master/passthrough-route.json route "route-secure" created service "hello-nginx-https" created pod "hello-nginx-docker-1" created pod "hello-nginx-docker-2" created [root@dhcp-41-211 ~]# [root@dhcp-41-211 ~]# [root@dhcp-41-211 ~]# oc project p1 Now using project "p1" on server "https://dhcp-41-211.bos.redhat.com:8443". [root@dhcp-41-211 ~]# oc scale dc/p1-router --replicas=0 && oc scale dc/p1-router --replicas=1 deploymentconfig "p1-router" scaled deploymentconfig "p1-router" scaled [root@dhcp-41-211 ~]# sleep 30 [root@dhcp-41-211 ~]# oc project p1 Already on project "p1" on server "https://dhcp-41-211.bos.redhat.com:8443". [root@dhcp-41-211 ~]# oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION hello-route hello-openshift.com hello-service <all> [root@dhcp-41-211 ~]# oc project p2 Now using project "p2" on server "https://dhcp-41-211.bos.redhat.com:8443". [root@dhcp-41-211 ~]# oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION route-reen reen.example.com service-secure https reencrypt [root@dhcp-41-211 ~]# oc project p3 Now using project "p3" on server "https://dhcp-41-211.bos.redhat.com:8443". [root@dhcp-41-211 ~]# oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION route-secure www.example.com hello-nginx-https <all> passthrough [root@dhcp-41-211 ~]# curl --resolve hello-openshift.com:80:$IP http://hello-openshift.com Hello OpenShift! [root@dhcp-41-211 ~]# curl --resolve reen.example.com:443:$IP https://reen.example.com --cacert route.pem Hello-OpenShift-1 https-8443 [root@dhcp-41-211 ~]# curl --resolve www.example.com:443:$IP https://www.example.com --cacert route.pem Hello World [root@dhcp-41-211 ~]# From router node: [root@dhcp-41-239 ~]# cid=$(docker ps | egrep "openshift3/(origin|ose)-haproxy-router" | awk '{print $1}') [root@dhcp-41-239 ~]# sudo nsenter -m -u -n -i -p -t $(docker inspect --format "{{ .State.Pid }}" "$cid") [root@dhcp-41-239 /]# cd /var/lib/haproxy/conf [root@dhcp-41-239 conf]# more /var/lib/haproxy/conf/os_http_be.map hello-openshift.com p1_hello-route [root@dhcp-41-239 conf]# ls default_pub_keys.pem haproxy.config os_edge_http_be.map os_edge_http_redirect.map os_reencrypt.map os_tcp_be.map error-page-503.http haproxy-config.template os_edge_http_expose.map os_http_be.map os_sni_passthrough.map [root@dhcp-41-239 conf]# more os_reencrypt.map reen.example.com p2_route-reen [root@dhcp-41-239 conf]# more os_sni_passthrough.map www.example.com 1 Actual results: [root@dhcp-41-239 conf]# more os_sni_passthrough.map www.example.com 1 Expected results: [root@dhcp-41-239 conf]# more os_sni_passthrough.map www.example.com p3-route-secure Additional info:
First, this should not be a problem which related to the router sharding. The contents of os_sni_passthrough.map file will be always like this. From the haproxy router template: {{/* os_sni_passthrough.map: contains a mapping of routes that expect to have an sni header and should be passed through to the host_be. Driven by the termination type of the ServiceAliasConfigs */}} {{ define "/var/lib/haproxy/conf/os_sni_passthrough.map" }} {{ range $idx, $cfg := .State }} {{ if and (eq $cfg.Path "") (eq $cfg.TLSTermination "passthrough") }} {{$cfg.Host}} 1 {{ end }} {{ end }} {{ end }}{{/* end sni passthrough map template */}} The "1" in the file looks like an indicator.
From end user point of view, what does "1" mean? It make no sense to customers. for consistency purpose, passthrough route in router sharding setup should display the same meaningful result as reencrypt and edge route did.
hi, Weibin I'm not sure if you already found any issues about function of passthrough route unless the content of this file is not perfect. for me, I don't think the customers I mean admin user here care about what the content is and also the normal user have not the right to view it. since the function of passthrough is working well. so I think this should not be an issue too.
If there two services using two different passthrough routes in two different namespaces: I hope the correct route entries should be like: www.example1.com 1 p1-route1 www.example2.com 1 p2-route2 The current route entries are like this: www.example1.com 1 www.example2.com 1 I am not sure how www.example1.com and www.example2.com know which namespace and which route it should use. I haven't setup to test above config, just curious
Jake, can you please see why we are writing 1 here rather than {{$idx}} for the mapping file? It seems to be the only one where it is different. This is not a user-visible change, so I'm lowering the priority.
The "1" can be anything, the haproxy template uses it as an if-exists check. Using 1 reduces the size of the map file, whether we care about the size of the map file could be a topic of discussion.