Description of problem: Unexpected iptables rules are saved to "/etc/sysconfig/iptables" on only first master as follows. Added iptables rules are related with SDN. * first master ~~~ :INPUT ACCEPT [0:0] :FORWARD DROP [0:0] :OUTPUT ACCEPT [21:2268] :DOCKER - [0:0] :DOCKER-ISOLATION - [0:0] :KUBE-EXTERNAL-SERVICES - [0:0] :KUBE-FIREWALL - [0:0] :KUBE-FORWARD - [0:0] :KUBE-NODEPORT-NON-LOCAL - [0:0] :KUBE-SERVICES - [0:0] :OPENSHIFT-ADMIN-OUTPUT-RULES - [0:0] :OPENSHIFT-FIREWALL-ALLOW - [0:0] :OPENSHIFT-FIREWALL-FORWARD - [0:0] :OS_FIREWALL_ALLOW - [0:0] -A INPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes externally-visible service portals" -j KUBE-EXTERNAL-SERVICES -A INPUT -m comment --comment "Ensure that non-local NodePort traffic can flow" -j KUBE-NODEPORT-NON-LOCAL -A INPUT -m comment --comment "firewall overrides" -j OPENSHIFT-FIREWALL-ALLOW -A INPUT -j KUBE-FIREWALL -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -j OS_FIREWALL_ALLOW -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -m comment --comment "kubernetes forwarding rules" -j KUBE-FORWARD -A FORWARD -i tun0 ! -o tun0 -m comment --comment "administrator overrides" -j OPENSHIFT-ADMIN-OUTPUT-RULES -A FORWARD -m comment --comment "firewall overrides" -j OPENSHIFT-FIREWALL-FORWARD -A FORWARD -j DOCKER-ISOLATION -A FORWARD -o docker0 -j DOCKER -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i docker0 ! -o docker0 -j ACCEPT -A FORWARD -i docker0 -o docker0 -j ACCEPT -A FORWARD -j REJECT --reject-with icmp-host-prohibited -A OUTPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes service portals" -j KUBE-SERVICES -A OUTPUT -j KUBE-FIREWALL -A DOCKER-ISOLATION -j RETURN -A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP -A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x1/0x1 -j ACCEPT -A KUBE-FORWARD -s 10.128.0.0/14 -m comment --comment "kubernetes forwarding conntrack pod source rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A KUBE-FORWARD -d 10.128.0.0/14 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A OPENSHIFT-FIREWALL-ALLOW -p udp -m udp --dport 4789 -m comment --comment "VXLAN incoming" -j ACCEPT -A OPENSHIFT-FIREWALL-ALLOW -i tun0 -m comment --comment "from SDN to localhost" -j ACCEPT -A OPENSHIFT-FIREWALL-ALLOW -i docker0 -m comment --comment "from docker to localhost" -j ACCEPT -A OPENSHIFT-FIREWALL-FORWARD -s 10.128.0.0/14 -m comment --comment "attempted resend after connection close" -m conntrack --ctstate INVALID -j DROP -A OPENSHIFT-FIREWALL-FORWARD -d 10.128.0.0/14 -m comment --comment "forward traffic from SDN" -j ACCEPT -A OPENSHIFT-FIREWALL-FORWARD -s 10.128.0.0/14 -m comment --comment "forward traffic to SDN" -j ACCEPT : ~~~ * Other master hosts ~~~ # Generated by iptables-save v1.4.21 on Sun May 12 21:15:13 2019 *filter :INPUT ACCEPT [0:0] :FORWARD DROP [0:0] :OUTPUT ACCEPT [0:0] :DOCKER - [0:0] :DOCKER-ISOLATION - [0:0] :OS_FIREWALL_ALLOW - [0:0] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -j OS_FIREWALL_ALLOW -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -j DOCKER-ISOLATION -A FORWARD -o docker0 -j DOCKER -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i docker0 ! -o docker0 -j ACCEPT -A FORWARD -i docker0 -o docker0 -j ACCEPT -A FORWARD -j REJECT --reject-with icmp-host-prohibited -A DOCKER-ISOLATION -j RETURN -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 10010 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 10250 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 10256 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT -A OS_FIREWALL_ALLOW -p udp -m state --state NEW -m udp --dport 4789 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 9000:10000 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 1936 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 2379 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 2380 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 8443 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 8444 -j ACCEPT -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 8053 -j ACCEPT -A OS_FIREWALL_ALLOW -p udp -m state --state NEW -m udp --dport 8053 -j ACCEPT COMMIT # Completed on Sun May 12 21:15:13 2019 # Generated by iptables-save v1.4.21 on Sun May 12 21:15:13 2019 *nat :PREROUTING ACCEPT [1420:153726] :INPUT ACCEPT [3:180] :OUTPUT ACCEPT [9248:649431] :POSTROUTING ACCEPT [9248:649431] :DOCKER - [0:0] :KUBE-MARK-MASQ - [0:0] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE -A DOCKER -i docker0 -j RETURN COMMIT # Completed on Sun May 12 21:15:13 2019 ~~~ Version-Release number of the following components: rpm -q openshift-ansible openshift-ansible-3.11.154-2.git.0.1640c49.el7.noarch rpm -q ansible ansible-2.6.14-1.el7ae.noarch ansible --version ansible 2.6.14 config file = /usr/share/ansible/openshift-ansible/ansible.cfg configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] How reproducible: You can always reproduce to install openshift 3.11 initially. Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Some sdn iptables rules are saved to "/etc/sysconfig/iptables" on only first master host. Expected results: SDN rules are not saved to "/etc/sysconfig/iptables" like other master hosts. Additional info: Please attach logs from ansible-playbook with the -vvv flag This issue cause is to run cockpit roles after running SDN roles as follows. Since cockpit roles has configurations, such as adding port and save the rules to "/etc/sysconfig/iptables". * playbooks/openshift-master/private/additional_config.yml ~~~ roles: # TODO: this is currently required in order to schedule pods onto the masters, but # should be moved into components once nodes are using dynamic config - role: openshift_sdn when: openshift_use_openshift_sdn | default(True) | bool - role: openshift_project_request_template when: openshift_project_request_template_manage - role: openshift_examples when: - openshift_install_examples | default(true) | bool - role: openshift_cluster_autoscaler when: openshift_cluster_autoscaler_deploy | default(false) | bool - role: openshift_manageiq when: openshift_use_manageiq | default(true) | bool - role: cockpit when: - not openshift_is_atomic | bool - openshift_deployment_type == 'openshift-enterprise' - osm_use_cockpit is undefined or osm_use_cockpit | bool - (openshift_deployment_subtype | default('')) != 'registry' : ~~~
Hi team, I've created PR here: https://github.com/openshift/openshift-ansible/pull/12052
Could reproduce this issue with openshift-ansible-3.11.161-1.git.0.376158f.el7.noarch.rpm When osm_use_cockpit=true, setup a 3-master ha cluster, check /etc/sysconfig/iptables on the first master [root@ip-172-18-1-124 ~]# grep SDN /etc/sysconfig/iptables -A OPENSHIFT-FIREWALL-ALLOW -i tun0 -m comment --comment "from SDN to localhost" -j ACCEPT -A OPENSHIFT-FIREWALL-FORWARD -d 10.2.0.0/16 -m comment --comment "forward traffic from SDN" -j ACCEPT -A OPENSHIFT-FIREWALL-FORWARD -s 10.2.0.0/16 -m comment --comment "forward traffic to SDN" -j ACCEPT The other two masters doesn't have SDN related rules in /etc/sysconfig/iptables. With openshift-ansible-3.11.165-1.git.0.2b41335.el7.noarch.rpm used, all three masters doesn't have such SDN rules saved in /etc/sysconfig/iptables.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0402