Bug 1143991 - [2.1 backport] Expose haproxy-sni-proxy mapped ports as environmental variables
Summary: [2.1 backport] Expose haproxy-sni-proxy mapped ports as environmental variables
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: ImageStreams
Version: 2.1.0
Hardware: All
OS: All
high
high
Target Milestone: ---
: ---
Assignee: Luke Meyer
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks: 1144788
TreeView+ depends on / blocked
 
Reported: 2014-09-18 12:06 UTC by Grzegorz Grzybek
Modified: 2015-04-03 15:38 UTC (History)
10 users (show)

Fixed In Version: rubygem-openshift-origin-node-1.23.9.26-1.el6op
Doc Type: Bug Fix
Doc Text:
Cartridge environment variables were created in new applications for TLS private ports but not for TLS public SNI proxy ports. This issue made it difficult to obtain application endpoint details that could be required by external services. This bug fix backports an OpenShift Enterprise 2.2 fix to expose the SNI proxy mapped ports as environment variables, and TLS public port information is now more readily available in new applications.
Clone Of:
: 1144788 (view as bug list)
Environment:
Last Closed: 2014-11-25 18:19:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker ENTESB-1952 0 Major Closed OSE PortMapper does not translate tls ports 2016-11-07 20:50:01 UTC
Red Hat Product Errata RHSA-2014:1906 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Enterprise 2.1.9 security, bug fix, and enhancement update 2014-11-25 23:19:05 UTC

Description Grzegorz Grzybek 2014-09-18 12:06:22 UTC
Description of problem:
In Fuse cartridge there are port mappings in the form of:

...
  - Private-IP-Name:   IP
    Private-Port-Name: APP_PORT_1
    Private-Port:      3001
    Public-Port-Name:  APP_PORT_1_PROXY_PORT
    Options:           { "ssl_to_gear": true }
...
  - Private-IP-Name:   IP
    Private-Port-Name: APP_TLS_PORT_1
    Private-Port:      4001
    Public-Port-Name:  APP_TLS_PORT_1_PROXY_PORT
    Protocols: [tls]
    Mappings:
    - Frontend: 'TLS_PORT_6'
      Backend: ''
...

for APP_PORT_1 we get environmental variables:
* OPENSHIFT_FUSE_APP_PORT_1
* OPENSHIFT_FUSE_APP_PORT_1_PROXY_PORT

But for APP_TLS_PORT_1 we only get APP_TLS_PORT_1.

Version-Release number of selected component (if applicable):
2.1.4


How reproducible:
always

Steps to Reproduce:
1. Install fuse cart from RPM (e.g. openshift-origin-cartridge-fuse-6.1.0.redhat.396-1.el6op)
2. Create application based on this cartridge
3. Check /var/lib/openshift/<ID>/.env directory

Actual results:
Files:
OPENSHIFT_FUSE_APP_TLS_PORT_1
OPENSHIFT_FUSE_APP_TLS_PORT_2
OPENSHIFT_FUSE_APP_TLS_PORT_3
OPENSHIFT_FUSE_APP_TLS_PORT_1_PROXY_PORT
OPENSHIFT_FUSE_APP_TLS_PORT_2_PROXY_PORT
OPENSHIFT_FUSE_APP_TLS_PORT_3_PROXY_PORT


Expected results:
OPENSHIFT_FUSE_APP_TLS_PORT_1
OPENSHIFT_FUSE_APP_TLS_PORT_2
OPENSHIFT_FUSE_APP_TLS_PORT_3


Additional info:

Comment 2 Luke Meyer 2014-09-18 15:08:58 UTC
Not clear how hard this is or whether it is a blocker for anything in 2.2. We definitely want to investigate both.

Comment 3 Ben Parees 2014-09-18 15:14:45 UTC
Grzegorz, what's the use case for those env variables?  The value would be one of the frontend SNI ports (eg 2303), but the app isn't going to try to reach itself over that port, so not sure why it needs that information?

Comment 4 Ben Parees 2014-09-18 15:15:32 UTC
Also I think your expected and actual results are reversed?

Comment 5 Marek Schmidt 2014-09-18 15:26:22 UTC
You need these e.g. for the fabric-cxf-registry, where you can register endpoints for each service, which can be consumed, e.g. by some load balancer, or a client configured to get the actual URL from the fabric by a logical name.

Comment 6 Ben Parees 2014-09-18 17:02:39 UTC
The assigned ports are reported when you create the app, is that sufficient for now?

Comment 7 Grzegorz Grzybek 2014-09-19 06:15:45 UTC
Ben:
 - right - application doesn't connect to itself via these ports
 - right - I reversed actual and expected (can I edit the description in BZ?)

As Marek said, Fuse is registering endpoints in ZooKeeper which are then used by external clients, which have to know external tls ports. We provide internal "port mapper" which translates internal (bind address + private port) to public (app-domain.host : tls port) addresses of WebServices/REST endpoints.

Ben - what do you mean by "assigned ports are reported"? I though about cartridge notifications, but couldn't find correct example of the event to which cart should be subscribed.

Comment 8 Ben Parees 2014-09-19 13:34:22 UTC
I just meant they are reported to the user during application create.  Though it's possible that same mechanism could be leveraged to get env variables created.

Comment 9 Ben Parees 2014-09-19 21:57:07 UTC
Working on a fix for this here:
https://github.com/openshift/origin-server/pull/5827

Comment 10 Ben Parees 2014-09-22 17:59:16 UTC
(upstream fix is in: https://github.com/openshift/origin-server/pull/5827)

Comment 11 Grzegorz Grzybek 2014-09-23 07:11:19 UTC
I applied the patch from origin-server and I added:

```
logger.info("REPORTED URLS: #{reported_urls}")
```
before iteration over `reported_urls` and I got the following log:

```
September 23 09:05:50 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [] => [127.13.36.129:8080] with options: {"protocols"=>["http"]}
September 23 09:05:51 INFO Shell command '/usr/sbin/httxt2dbm -f DB -i /etc/httpd/conf.d/openshift/nodes.txt -o /etc/httpd/conf.d/openshift/nodes.db-20140923-25644-1v5r413/new.db' ran. rc=0 out=
September 23 09:05:51 INFO REPORTED URLS: []
September 23 09:05:51 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_1] => [127.13.36.129:31314] with options: {"protocols"=>["tls"]}
September 23 09:05:52 INFO REPORTED URLS: []
September 23 09:05:52 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_2] => [127.13.36.129:31324] with options: {"protocols"=>["tls"]}
September 23 09:05:53 INFO REPORTED URLS: []
September 23 09:05:53 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_3] => [127.13.36.129:31334] with options: {"protocols"=>["tls"]}
September 23 09:05:53 INFO REPORTED URLS: []
September 23 09:05:53 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_4] => [127.13.36.129:31344] with options: {"protocols"=>["tls"]}
September 23 09:05:54 INFO REPORTED URLS: []
September 23 09:05:54 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_5] => [127.13.36.129:30304] with options: {"protocols"=>["tls"]}
September 23 09:05:54 INFO REPORTED URLS: []
September 23 09:05:54 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_6] => [127.13.36.129:4001] with options: {"protocols"=>["tls"]}
September 23 09:05:55 INFO REPORTED URLS: []
September 23 09:05:55 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_7] => [127.13.36.129:4002] with options: {"protocols"=>["tls"]}
September 23 09:05:56 INFO REPORTED URLS: []
September 23 09:05:56 INFO Connecting frontend mapping for 54211b80e659c5a57b00000e/fuse: [TLS_PORT_8] => [127.13.36.129:4003] with options: {"protocols"=>["tls"]}
September 23 09:05:56 INFO REPORTED URLS: []
```

The environmental variables for proxy ports were not created... It seems that SNI plugin don't return the reported_urls...

Comment 17 xjia 2014-11-06 08:11:04 UTC
Puddle:
2.1.z/2014-11-05.1

Verify:
Create a fuse application:
[fuse-domainwyy.ose21z-manual.com.cn 545b2bf8db26c8f63f000268]\> env | grep OPENSHIFT_FUSE_APP_TLS_PORT
OPENSHIFT_FUSE_APP_TLS_PORT_1_PROXY_PORT=2308
OPENSHIFT_FUSE_APP_TLS_PORT_3_PROXY_PORT=2310
OPENSHIFT_FUSE_APP_TLS_PORT_1=4001
OPENSHIFT_FUSE_APP_TLS_PORT_3=4003
OPENSHIFT_FUSE_APP_TLS_PORT_2=4002
OPENSHIFT_FUSE_APP_TLS_PORT_2_PROXY_PORT=2309

Comment 19 errata-xmlrpc 2014-11-25 18:19:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2014-1906.html

Comment 20 JBoss JIRA Server 2015-01-09 16:23:10 UTC
Hiram Chirino <hiram> updated the status of jira ENTESB-1952 to Resolved

Comment 21 JBoss JIRA Server 2015-03-13 11:46:20 UTC
Marek Schmidt <maschmid> updated the status of jira ENTESB-1952 to Reopened

Comment 22 JBoss JIRA Server 2015-03-24 20:54:11 UTC
Hiram Chirino <hiram> updated the status of jira ENTESB-1952 to Resolved

Comment 23 JBoss JIRA Server 2015-04-03 15:38:24 UTC
Marek Schmidt <maschmid> updated the status of jira ENTESB-1952 to Closed


Note You need to log in before you can comment on or make changes to this bug.