Bug 1268904

Was OOO - just got back today. Yeah, will have to address it in 3.2

Just to clarify then:

(In reply to Miheer Salunke from comment #4)

> And it would be nice to have:
> 
>  ports:
> - containerPort: 8080
>   hostPort: 8080
>   protocol: TCP
>  - containerPort: 8443
>   hostPort: 8443
>   protocol: TCP
> 
> And using host network.

So this request is an RFE to have the router image's ports configurable somehow, right? Updating component accordingly

@Harald, was on vacation. Just got back so am not certain what the 3.2 release schedule or dates are but I created a PR for this. 
See: https://github.com/openshift/origin/pull/7704

Setting to ON_QA 
To test create and edit the router deployment config:
   $ oadm router --latest-images

   $ oc edit dc router
   
   #  and set the ROUTER_SERVICE_HTTP_PORT and ROUTER_SERVICE_HTTPS_PORT environment
   #  variables appropriately. And also set the ports (container and host ports) to
   #  the same port values (example 10080 and 10443). 


Or alternatively, you could use:
   oadm router --latest-images --ports=... 
   oc env dc/router ROUTER_SERVICE_HTTP[S]_PORT=...
commands to set the ports + env values. 

Also note, there could potentially be other code ala the console (UI) and CLI code and external integrations that may have 80/443 hardcoded to check if a service is up and running, so this fix is specific for this use case to allow custom port numbers and allow multiple routers to be run with host-networking (use different port numbers).

Tested this issue on 
# oc version
oc v1.1.3-362-g7ba70f0
kubernetes v1.2.0-alpha.7-703-gbc4550d
openshift/origin-haproxy-router          latest              8391e00c75f7

scenario steps:
1. Create multi-router with different port with host-networking and container networking

# oc get endpoints first second third
NAME      ENDPOINTS                                                 AGE
first     172.18.14.20:10080,172.18.14.20:10443,172.18.14.20:1936   1h
second    172.18.14.20:80,172.18.14.20:443,172.18.14.20:1937        30m
third     172.17.0.4:1080,172.17.0.4:1938,172.17.0.4:1443           5m

2. create pod/service/route(unsecure,edge,passthrouth/reencryption)

the steps are work well

so this bug should has been fixed on origin.

since bug is reported on OSE and the code has not been merged to latest OSE. so in case it is forgot to be merged. I changed the status to 'assigned' for now. 
please help change back to 'ON_QA' once it was merged to OSE, I will verify it. thanks

sorry,should be 'modified'

hi, ram

I saw clayton stop the hostport port in https://github.com/openshift/origin/commit/241d02b8e89d492ad680559e56bb1549c9985c34

as my understanding, he want to use '--ports' to implement the mapping port between host and container when using container network mode.

and this https://github.com/openshift/origin/pull/7704 did not resolve this issue.

could you help confirm it, thanks.

@zhaozhanqi not sure I understand your test case. The default port numbers we
used in the past were always 80 and 443 irrespective of whether you run in
host-networking or container-networking mode.

Basically to test you'd run: 
oadm router --latest-images
oc env dc router ROUTER_SERVICE_HTTPS_PORT=5443 ROUTER_SERVICE_HTTP_PORT=5080

# cleanup router  
oadm router --latest-images --host-network=false
oc env dc router ROUTER_SERVICE_HTTPS_PORT=10443 ROUTER_SERVICE_HTTP_PORT=10080

The ports needing to be exposed in container networking mode are orthogonal to this PR. 
Meaning even without this PR, they need to be done.


And in the host networking case, these would match up automatically, example test case:
oadm router --latest-images
oc env dc router ROUTER_SERVICE_HTTPS_PORT=5443 ROUTER_SERVICE_HTTP_PORT=5080
curl -k -H "Host: allow-http.header.test" -H "now: $(date +%s)" http://127.0.0.1:5080

@Ram

yes, before for container networking the router can export 'hostPort', thus container and host will have a mapping port,example:
# docker ps
CONTAINER ID        IMAGE                                                                                           COMMAND                  CREATED             STATUS              PORTS                                                              NAMES
b53732ae460b        brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/ose-haproxy-router:v3.1.1.903   "/usr/bin/openshift-r"   2 days ago          Up 2 days                                                                              k8s_router.c86b40c7_router-1-723p8_default_c83ddf2f-e1d7-11e5-ac77-525400719108_6814bfb9
5dc1dcbc6c63        brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/ose-pod:v3.1.1.903              "/pod"                   2 days ago          Up 2 days           0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:1936->1936/tcp   k8s_POD.9629577a_router-1-723p8_default_c83ddf2f-e1d7-11e5-ac77-525400719108_422aa0c2


you can see the mapping info: 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:1936->1936/tcp 

then we can also access the router using host port.

but now after the https://github.com/openshift/origin/commit/241d02b8e89d492ad680559e56bb1549c9985c34
rotuer.go #line 414

the 'hostPort' has been deleted. then the container are:

# docker ps
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS              PORTS               NAMES
5017f2393100        openshift/origin-haproxy-router:latest   "/usr/bin/openshift-r"   8 minutes ago       Up 8 minutes                            k8s_router.706c7f85_router-1-7892t_default_20c1bb85-e408-11e5-8528-0e754fd826a7_4faa5e5a
bc6c09a5d360        openshift/origin-pod:v1.1.3              "/pod"                   9 minutes ago       Up 8 minutes                            k8s_POD.909543fa_router-1-7892t_default_20c1bb85-e408-11e5-8528-0e754fd826a7_72461cb1

you can see NO mapping port between host and container.
since router container ip can NOT be accessed out of openshift cluster

So I'm wondering how to resolve the router container ip from out of openshift cluster.
I thought this will be resolve using '--ports'.

@Ram

Comment 13 issue can be resolved via setting 'externalIPs' in the router service. 
I will verified this bug once it's merged to OSE

@zhaozhanqi - thx for the update. Cool yeah this change was just to allow the port number to be configured.
 
The changes to container networking Clayton made were aside from this fix and now require the port to be exposed - example via a node-port service. Glad you figured that out.

@Ram Ranganathan

for hostnetwork mode router, if customer is using the specified port: eg 1081/10444.  if so, we need to tell user add iptables or Do we have another method?


steps:

1. Create router using --ports

 oadm router second --credentials=/etc/origin/master/openshift-router.kubeconfig --service-account=router --images='openshift3/ose-${component}:${version}' --stats-port=6701 --ports='1081:1081,10444:10444'

2. oc env dc second ROUTER_SERVICE_HTTP_PORT=1081
   oc env dc second ROUTER_SERVICE_HTTP_PORTS=10443

3. return no route when accessing the nodeport
   # curl 10.36.136.222:1081
curl: (7) Failed connect to 10.36.136.222:1081; Connection refused

@zhaozhanqi - so this is assuming the nodes (where the router pods run) have those ports open - similar to the case for 80/443. Otherwise, no traffic is going to get in. 
I guess, we can probably add a section to the docs to mentioning to how to run routers on custom ports. Will send a PR to update the docs sometime later this week. Thx

Added the docs changes earlier - forgot to update the bugz here. 
The docs PR was: https://github.com/openshift/openshift-docs/pull/1942 
and has been merged.

@zhaozhanqi FYI.

@Ram

In this line:
https://github.com/openshift/openshift-docs/pull/1942/files#diff-7c33c875858cf1c9cf05f070744861faR247

should add option 'ports' when creating router ,like 

oadm router --replicas=0 --ports='10080:10080,10443:10443' 

otherwise, 

the endpoints will still 80 and 443

# oc get endpoints router
router             10.66.136.222:80,10.66.136.222:443,10.66.136.222:1936   22m

that's mean the hostport and container port still 80/443. 

and I doubt this will make container network router can not work

@zhaozhanqi, true that we can clarify it a bit more (that example is for host networking but point taken re: endpoints info mismatch).
Just updated the example. PR: https://github.com/openshift/openshift-docs/pull/2115

@Ram

thanks, it looks good to me.

This has been merged into ose and is in OSE v3.3.0.28 or newer.

Checked on router image openshift3/ose-haproxy-router:v3.3.0.28 6ff97e2fd698

Issue has been fixed.

The ports can be modified via env ROUTER_SERVICE_HTTP_PORT ROUTER_SERVICE_HTTPS_PORT ROUTER_SERVICE_SNI_PORT ROUTER_SERVICE_NO_SNI_PORT.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933