Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1415800 - Installer fails to add/check iptables rule due to lock on xtables.
Installer fails to add/check iptables rule due to lock on xtables.
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer (Show other bugs)
3.4.0
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: Andrew Butcher
Wenkai Shi
:
Depends On:
Blocks: 1445194
  Show dependency treegraph
 
Reported: 2017-01-23 13:34 EST by Ryan Howe
Modified: 2017-07-24 10 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously the installer may have failed to add iptables rules if other iptables rules were being updated at the same time. Now the installer waits to obtain a lock when updating iptables rules ensuring that rules are properly created.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-04-12 14:49:12 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0903 normal SHIPPED_LIVE OpenShift Container Platform atomic-openshift-utils bug fix and enhancement 2017-04-12 18:45:42 EDT

  None (edit)
Description Ryan Howe 2017-01-23 13:34:32 EST
Description of problem:

When running the playbooks, the installer will fail during the os_firewall : Add iptables allow rules role/task due to a lock on xtables. 


Version-Release number of selected component (if applicable):
OpenShift 3.x

How reproducible:
Random 

Additional info:

Kubernetes hit this issue with OpenShift seen here in upstream issue:  https://github.com/kubernetes/kubernetes/issues/7370

The solution could be to add the -w flag to  wait for the xtables lock in the os_firewall_manage_iptables module.  

https://github.com/openshift/openshift-ansible/blob/release-1.4/roles/os_firewall/library/os_firewall_manage_iptables.py


######################
ERROR:
failed: [1_2_6_1] (item={u'port': u'10250/tcp', u'service': u'Kubernetes kubelet'}) => {
    "failed": true, 
    "invocation": {
        "module_args": {
            "action": "add", 
            "chain": "OS_FIREWALL_ALLOW", 
            "create_jump_rule": true, 
            "ip_version": "ipv4", 
            "jump_rule_chain": "INPUT", 
            "name": "Kubernetes kubelet", 
            "port": "10250", 
            "protocol": "tcp"
        }, 
        "module_name": "os_firewall_manage_iptables"
    }, 
    "item": {
        "port": "10250/tcp", 
        "service": "Kubernetes kubelet"
    }, 
    "msg": "Failed to create chain: OS_FIREWALL_ALLOW"
}

Checking if the rules generates a exit code not equal to 0 and then the module will try to create the chain or rule and then fail because it is already created. 

# iptables -C OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 10250  -j ACCEPT
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
# echo $?
4

# iptables -L OS_FIREWALL_ALLOW
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
# echo $?
4

# iptables -C INPUT  -j OS_FIREWALL_ALLOW 
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
# echo $?
4
Comment 1 openshift-github-bot 2017-01-24 09:52:46 EST
Commit pushed to master at https://github.com/openshift/openshift-ansible

https://github.com/openshift/openshift-ansible/commit/182a07ad58972cddb3747a18d6a8e3818492488c
Merge pull request #3152 from abutcher/iptables-w

Bug 1415800 - Installer fails to add/check iptables rule due to lock on xtables.
Comment 2 Wenkai Shi 2017-02-03 05:18:37 EST
Verified with version openshift-ansible-3.5.3-1.
Currently installer use firewalld modules rather than iptables to manage firewall by default, add "os_firewall_use_firewalld=false" to make sure installer use iptables. 
During installation, run a while loop of iptables command on hosts, installer will waiting at "TASK [os_firewall : Add iptables allow rules]". Once stop the while loop, installer continue.

[root@master ~]# while true; do iptables -nL; done
...

[root@ansible ~]# ansible-playbook -i hosts -v /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml
...
TASK [os_firewall : Add iptables allow rules] **********************************
...
(waiting here during the "while loop" alive)
...
Comment 5 errata-xmlrpc 2017-04-12 14:49:12 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0903

Note You need to log in before you can comment on or make changes to this bug.