Bug 1027122

I talked with the system-config-firewall maintainer about this a bit and here's a possible solution:

* OSE install could modify /etc/sysconfig/system-config-firewall and add: 

--custom-rules=ipv4:filter:/etc/openshift/iptables.filter.rules
--custom-rules=ipv4:nat:/etc/openshift/iptables.nat.rules

This would cause the rules to be added just before the right before the reject rules at the end.  This is slightly different than today where we inject '-A INPUT -j rhc-app-comm' before the location where admins would normally make their customizations.  This _shouldn't_ be a problem because OpenShift gear ports shouldn't conflict with common system services (if they do there will be other problems)

* We still have a problem with making sure lokkit doesn't nuke the chain.  I think we would add a new custom-rule file that would handle the chain creation.

* There is testing needed with 'service iptables save'.  If an admin were to run that then the chain creation would be written to /etc/sysconfig/iptables.  When lokkit tries to readd the chain it will fail.

I added the following to /etc/sysconfig/system-config-firewall:

--custom-rules=ipv4:filter:/etc/openshift/system-config-firewall
--custom-rules=ipv4:filter:/etc/openshift/iptables.filter.rules
--custom-rules=ipv4:nat:/etc/openshift/iptables.nat.rules

The content of /etc/openshift/system-config-firewall is:

-N rhc-app-comm
-A INPUT -j rhc-app-comm

With this in place the following works as expected:

# manage /etc/sysconfig/iptables with lokkit and overwrite/reload that file
$ lokkit --service=ssh

# see if our chain still exists
$ service openshift-iptables-port-proxy status
The OpenShift iptables port proxy is enabled.

# stop the chain
$ service openshift-iptables-port-proxy stop
$ service openshift-iptables-port-proxy status
ERROR: A difference has been detected between state of /etc/openshift/iptables.filter.rules and the rhc-app-comm iptables chain.

# try again and verify it loads the chain back
$ lokkit --service=ssh
$ service openshift-iptables-port-proxy status
The OpenShift iptables port proxy is enabled.

# verify iptables-save works as expected
$ service iptables save
$ lokkit --service=ssh
$service openshift-iptables-port-proxy status
The OpenShift iptables port proxy is enabled.

I think we can consider putting the needed logic in openshift.sh.  Alternatively we sould add it to the rubygem-openshift-origin-node package.  I'd prefer to do the latter upstream and test it first.

We need to verify that oo-diagnostics catches the missing iptables chain.

For clarification, the proposed fix for this is to advise admins not to use lokkit.  We will detect that in oo-diagnostics.  However, since it ships with RHEL we will do our best to maintain compatibility.  To that in oo-diagnostics will also provide advice on how to configure OSE to work with lokkit (as best it can).

verified with puddle-2-0-3-2014-01-30

[root@broker sysconfig]# oo-diagnostics 
WARN: test_node_profiles_districts_from_broker
        No districts are defined. Districts should be used in any production installation.
        Please consult the Administration Guide.

FAIL: test_services_enabled
      The following service(s) are not currently started:
        openshift-iptables-port-proxy
      These services are required for OpenShift functionality.

WARN: test_system_config_firewall
         Using system-config-firewall and lokkit with OpenShift is not recommended.
         To continue using lokkit please ensure the following custom rules are 
         installed in /etc/sysconfig/system-config-firewall:

         --custom-rules=ipv4:filter:/etc/openshift/system-config-firewall-compat
         --custom-rules=ipv4:filter:/etc/openshift/iptables.filter.rules
         --custom-rules=ipv4:nat:/etc/openshift/iptables.nat.rules

WARN: test_altered_package_owned_configs
          The mlocate package is not installed. mlocate is not a required runtime package; however,
          you may install mlocate to enable further diagnostics checking.

WARN: test_broker_certificate
Using a self-signed certificate for the broker
WARN: test_yum_configuration
        oo-admin-yum-validator reported some possible problems
        with your package source configuration:
--------------------------------------------------------------
      No roles have been specified. Attempting to guess the roles for this system...
If the roles listed below are incorrect or incomplete, please re-run this script with the appropriate --role arguments
    node
    broker
    client
If this system will be providing the JBossEAP cartridge, re-run this command with the --role=node-eap argument
Detected OpenShift Enterprise repository subscription managed by Red Hat Subscription Manager.
Detected installed OpenShift Enterprise version 2.0

Checking if yum-plugin-priorities is installed
Checking channel/repository priorities
Resolving repository/channel/subscription priority conflicts
To resolve conflicting repositories, update repo priority by running:
# yum-config-manager --setopt=openshift_client_tools_extra.priority=40 openshift_client_tools_extra --save
# yum-config-manager --setopt=openshift_node_extra.priority=40 openshift_node_extra --save
# yum-config-manager --setopt=openshift_infra_extra.priority=40 openshift_infra_extra --save
Please re-run this tool after making any recommended repairs to this system

--------------------------------------------------------------
        Incorrect package source configuration could lead to
        failure to install the correct RPMs.

5 WARNINGS
1 ERRORS

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0209.html