Bug 1501855 - the rules added in chain OPENSHIFT-ADMIN-OUTPUT-RULES cannot work
Summary: the rules added in chain OPENSHIFT-ADMIN-OUTPUT-RULES cannot work
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.7.0
Assignee: Dan Winship
QA Contact: Meng Bo
URL:
Whiteboard:
: 1505897 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-13 10:57 UTC by Hongan Li
Modified: 2017-11-28 22:17 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-11-28 22:17:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Description Hongan Li 2017-10-13 10:57:37 UTC
Description of problem:
the rules added in chain OPENSHIFT-ADMIN-OUTPUT-RULES cannot work

Version-Release number of selected component (if applicable):
openshift v3.7.0-0.147.1
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.1

How reproducible:
always

Steps to Reproduce:
1. create project and pod.
 
2. add rule in chain OPENSHIFT-ADMIN-OUTPUT-RULES to reject all outbouding traffic from the pod.
# iptables -A OPENSHIFT-ADMIN-OUTPUT-RULES -s 10.129.0.27 -j REJECT

3. oc rsh pod and curl website.

Actual results:
still can reach outside network from this pod


Expected results:
should not reach outside network since admin add reject rule in OPENSHIFT-ADMIN-OUTPUT-RULES

Additional info:
the function works in OCP 3.6 and iptables like this:
-A FORWARD -i tun0 ! -o tun0 -m comment --comment "administrator overrides" -j OPENSHIFT-ADMIN-OUTPUT-RULES
-A FORWARD -m comment --comment "firewall overrides" -j OPENSHIFT-FIREWALL-FORWARD

but the sequence is changed in OCP 3.7 as below:
-A FORWARD -m comment --comment "firewall overrides" -j OPENSHIFT-FIREWALL-FORWARD
-A FORWARD -i tun0 ! -o tun0 -m comment --comment "administrator overrides" -j OPENSHIFT-ADMIN-OUTPUT-RULES

Comment 1 Dan Winship 2017-10-16 15:00:20 UTC
https://github.com/openshift/origin/pull/16884

Comment 3 Hongan Li 2017-10-23 07:11:51 UTC
verified in atomic-openshift-3.7.0-0.173.0.git.0.cf24e11 and issue has been fixed.

# iptables -nL
...<snip>...
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
OPENSHIFT-ADMIN-OUTPUT-RULES  all  --  0.0.0.0/0            0.0.0.0/0            /* administrator overrides */
OPENSHIFT-FIREWALL-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* firewall overrides */

Comment 4 Meng Bo 2017-10-23 08:06:31 UTC
The fix introduced a new problem when using networkpolicy plugin.

The node cannot be started due to the iptables chain OPENSHIFT-MASQUERADE-2 does not exist.


Oct 23 15:40:32 ose-node2.bmeng.local atomic-openshift-node[39442]: F1023 15:40:32.035133   39442 network.go:45] SDN node startup failed: failed to set up iptables: failed to ensure rule [-s 10.128.0.0/14 -m comment --comment masquerade pod-to-external traffic -j OPENSHIFT-MASQUERADE-2] exists: error checking rule: exit status 2: iptables v1.4.21: Couldn't load target `OPENSHIFT-MASQUERADE-2':No such file or directory
Oct 23 15:40:32 ose-node2.bmeng.local atomic-openshift-node[39442]: Try `iptables -h' or 'iptables --help' for more information.
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: atomic-openshift-node.service: main process exited, code=exited, status=255/n/a
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: Failed to start Atomic OpenShift Node.
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: Unit atomic-openshift-node.service entered failed state.
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: atomic-openshift-node.service failed.

Comment 5 Dan Winship 2017-10-23 15:12:34 UTC
https://github.com/openshift/origin/pull/17001

Comment 7 Meng Bo 2017-10-25 05:54:47 UTC
*** Bug 1505897 has been marked as a duplicate of this bug. ***

Comment 8 Hongan Li 2017-10-30 05:34:33 UTC
verified in atomic-openshift-3.7.0-0.184.0.git.0.3bd6d30 and issue has been fixed.

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
OPENSHIFT-ADMIN-OUTPUT-RULES  all  --  0.0.0.0/0            0.0.0.0/0            /* administrator overrides */
OPENSHIFT-FIREWALL-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* firewall overrides */


Chain OPENSHIFT-MASQUERADE (1 references)
target     prot opt source               destination         
OPENSHIFT-MASQUERADE-2  all  --  10.128.0.0/14        0.0.0.0/0            /* masquerade pod-to-external traffic */

Chain OPENSHIFT-MASQUERADE-2 (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            10.128.0.0/14        /* masquerade pod-to-external traffic */
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

Comment 11 errata-xmlrpc 2017-11-28 22:17:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188


Note You need to log in before you can comment on or make changes to this bug.