Bug 1501855

Summary: the rules added in chain OPENSHIFT-ADMIN-OUTPUT-RULES cannot work
Product: OpenShift Container Platform Reporter: Hongan Li <hongli>
Component: NetworkingAssignee: Dan Winship <danw>
Status: CLOSED ERRATA QA Contact: Meng Bo <bmeng>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.7.0CC: aos-bugs, mifiedle, xtian
Target Milestone: ---Keywords: TestBlocker
Target Release: 3.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-28 22:17:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hongan Li 2017-10-13 10:57:37 UTC
Description of problem:
the rules added in chain OPENSHIFT-ADMIN-OUTPUT-RULES cannot work

Version-Release number of selected component (if applicable):
openshift v3.7.0-0.147.1
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.1

How reproducible:
always

Steps to Reproduce:
1. create project and pod.
 
2. add rule in chain OPENSHIFT-ADMIN-OUTPUT-RULES to reject all outbouding traffic from the pod.
# iptables -A OPENSHIFT-ADMIN-OUTPUT-RULES -s 10.129.0.27 -j REJECT

3. oc rsh pod and curl website.

Actual results:
still can reach outside network from this pod


Expected results:
should not reach outside network since admin add reject rule in OPENSHIFT-ADMIN-OUTPUT-RULES

Additional info:
the function works in OCP 3.6 and iptables like this:
-A FORWARD -i tun0 ! -o tun0 -m comment --comment "administrator overrides" -j OPENSHIFT-ADMIN-OUTPUT-RULES
-A FORWARD -m comment --comment "firewall overrides" -j OPENSHIFT-FIREWALL-FORWARD

but the sequence is changed in OCP 3.7 as below:
-A FORWARD -m comment --comment "firewall overrides" -j OPENSHIFT-FIREWALL-FORWARD
-A FORWARD -i tun0 ! -o tun0 -m comment --comment "administrator overrides" -j OPENSHIFT-ADMIN-OUTPUT-RULES

Comment 1 Dan Winship 2017-10-16 15:00:20 UTC
https://github.com/openshift/origin/pull/16884

Comment 3 Hongan Li 2017-10-23 07:11:51 UTC
verified in atomic-openshift-3.7.0-0.173.0.git.0.cf24e11 and issue has been fixed.

# iptables -nL
...<snip>...
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
OPENSHIFT-ADMIN-OUTPUT-RULES  all  --  0.0.0.0/0            0.0.0.0/0            /* administrator overrides */
OPENSHIFT-FIREWALL-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* firewall overrides */

Comment 4 Meng Bo 2017-10-23 08:06:31 UTC
The fix introduced a new problem when using networkpolicy plugin.

The node cannot be started due to the iptables chain OPENSHIFT-MASQUERADE-2 does not exist.


Oct 23 15:40:32 ose-node2.bmeng.local atomic-openshift-node[39442]: F1023 15:40:32.035133   39442 network.go:45] SDN node startup failed: failed to set up iptables: failed to ensure rule [-s 10.128.0.0/14 -m comment --comment masquerade pod-to-external traffic -j OPENSHIFT-MASQUERADE-2] exists: error checking rule: exit status 2: iptables v1.4.21: Couldn't load target `OPENSHIFT-MASQUERADE-2':No such file or directory
Oct 23 15:40:32 ose-node2.bmeng.local atomic-openshift-node[39442]: Try `iptables -h' or 'iptables --help' for more information.
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: atomic-openshift-node.service: main process exited, code=exited, status=255/n/a
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: Failed to start Atomic OpenShift Node.
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: Unit atomic-openshift-node.service entered failed state.
Oct 23 15:40:32 ose-node2.bmeng.local systemd[1]: atomic-openshift-node.service failed.

Comment 5 Dan Winship 2017-10-23 15:12:34 UTC
https://github.com/openshift/origin/pull/17001

Comment 7 Meng Bo 2017-10-25 05:54:47 UTC
*** Bug 1505897 has been marked as a duplicate of this bug. ***

Comment 8 Hongan Li 2017-10-30 05:34:33 UTC
verified in atomic-openshift-3.7.0-0.184.0.git.0.3bd6d30 and issue has been fixed.

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
OPENSHIFT-ADMIN-OUTPUT-RULES  all  --  0.0.0.0/0            0.0.0.0/0            /* administrator overrides */
OPENSHIFT-FIREWALL-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* firewall overrides */


Chain OPENSHIFT-MASQUERADE (1 references)
target     prot opt source               destination         
OPENSHIFT-MASQUERADE-2  all  --  10.128.0.0/14        0.0.0.0/0            /* masquerade pod-to-external traffic */

Chain OPENSHIFT-MASQUERADE-2 (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            10.128.0.0/14        /* masquerade pod-to-external traffic */
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

Comment 11 errata-xmlrpc 2017-11-28 22:17:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188