Bug 983475 - When node is installed on the same machine as broker, broker Apache configuration is ignored
When node is installed on the same machine as broker, broker Apache configura...
Status: CLOSED UPSTREAM
Product: OpenShift Origin
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Rob Millner
libra bugs
: UpcomingRelease
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-11 05:54 EDT by Jan Pazdziora
Modified: 2015-05-14 19:12 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-08-19 19:05:15 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jan Pazdziora 2013-07-11 05:54:29 EDT
Description of problem:

When /etc/httpd/conf.d/000002_openshift_origin_broker_proxy.conf is tweaked, the configuration is working when broker is installed on machien on its own. When node is installed on the same machine as well, /etc/httpd/conf.d/000001_openshift_origin_node.conf and /etc/httpd/conf.d/openshift_route.include take precedence even for the broker hostname.

Version-Release number of selected component (if applicable):

openshift-origin-broker-1.5.2-1.1.el6op.noarch
rubygem-openshift-origin-node-1.9.13-1.2.el6op.noarch

How reproducible:

Deterministic.

Steps to Reproduce:
1. cp /etc/pki/tls/certs/localhost.crt /var/www/html/CA.crt
2. chmod a+r /var/www/html/CA.crt
3. Apply the following patch to /etc/httpd/conf.d/000002_openshift_origin_broker_proxy.conf

diff --git a/broker/httpd/000002_openshift_origin_broker_proxy.conf b/broker/httpd/000002_openshift_origin_broker_proxy.conf
index c9bcd63..3649877 100644
--- a/broker/httpd/000002_openshift_origin_broker_proxy.conf
+++ b/broker/httpd/000002_openshift_origin_broker_proxy.conf
@@ -18,6 +18,7 @@
   ServerAdmin root@localhost
   DocumentRoot /var/www/html
   RewriteEngine              On
+  RewriteRule     ^/CA\.crt$    -       [L]
   RewriteRule     ^/$    https://%{HTTP_HOST}/console [R,L]
   RewriteRule     ^(.*)$     https://%{HTTP_HOST}$1 [R,L]
 </VirtualHost>

4. service httpd restart
5. GET http://$( hostname )/CA.crt

Actual results:

On broker-only machine, the certificate is shown. On node, Enter username for OpenShift Developer Console at ... is shown.

Expected results:

When the broker hostname is used in the URI, broker configuration should be respected, even if there are gears configured on that machine.

Additional info:

Filing this bugzilla based on thread http://lists.openshift.redhat.com/openshift-archives/dev/2013-July/msg00101.html.
Comment 1 Rob Millner 2013-07-31 14:05:27 EDT
I did some experimentation to see what we could do for this request.  There are a few different options for adding a file to be shared but its fairly limited.

1. Share the file from the broker's back-end Apache instead of the front-end Apache.

Ref: /var/www/openshift/broker/httpd/broker.conf


2.  Add the file to __default__ with a stanza like the following:

__default__/CA.crt FILE:/var/www/html/CA.crt


3. The default virtual host can be modified or an entirely separate one can be put in place.  

For example, devenv uses its own default virtual host definitions in 000000_default.conf that completely override what's in *_node.conf and *_broker.conf.  

Similarly, Online has their own *_default.conf.

I would treat this as an expert-level option, its very easy to break gear routing doing this.



Background:

The mod_rewrite router lives in the default (localhost, hostname) virtual host and we don't create virtual hosts for applications.  Apache only uses the first definition it reads for any virtual host (named by the ServerName).  While proxying from one virtual host to another is possible, jumping to the next definition of the same virtual host is not.

That's one of the trade-offs for the performance gains we achieved using mod_rewrite instead of virtual hosts - gear routing owns Apache.  

Our high density use cases have thousands of gears.  Apache with thousands of virtual hosts takes gigabytes of RAM and adds several minutes to the cost of app creation.  The mod_rewrite router scales to 100's of thousands of entries with negligible memory and time increases.

The __default__ route was given to Origin so that it could treat the broker as an application and not have to come up with its own customized 000000_default.conf to handle the combined case.


Side note:

Instructing people to install a self-created CA into their browsers is bad practice.  For example, it can be used to sign a forged etrade.com or fidelity.com site.  While we would never intentionally do that - its a door that's best left closed, locked, nailed shut, and bricked over.  8-)
Comment 2 Rob Millner 2013-07-31 14:08:20 EDT
Soliciting feedback.
Comment 3 Jan Pazdziora 2013-08-05 07:38:24 EDT
(In reply to Rob Millner from comment #1)
> 
> The __default__ route was given to Origin so that it could treat the broker
> as an application and not have to come up with its own customized
> 000000_default.conf to handle the combined case.

I think either being able to handle the custom-modifications as additional application or broker plugin is the way to go.

> Instructing people to install a self-created CA into their browsers is bad
> practice.  For example, it can be used to sign a forged etrade.com or
> fidelity.com site.  While we would never intentionally do that - its a door
> that's best left closed, locked, nailed shut, and bricked over.  8-)

Understood. The /CA.crt thing is just an example of customization that the admin might want to do to their installation. Running Nagios or IPA or some /var/www/html/pub content might be different one.
Comment 4 Rob Millner 2013-08-19 19:05:15 EDT
I've moved this request to a Trello Card to support both the mod_rewrite router for high density deployments and Virtual Hosts for deployments where customization is necessary.

https://trello.com/c/zw4iNj3g/245-frontendhttpserver-plugin-module-for-apache-virtual-hosts

Note You need to log in before you can comment on or make changes to this bug.