Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1489074

Summary: iptables manager may fail to apply firewall rules if another iptables* process is being executed
Product: Red Hat OpenStack Reporter: Ihar Hrachyshka <ihrachys>
Component: openstack-neutronAssignee: Ihar Hrachyshka <ihrachys>
Status: CLOSED ERRATA QA Contact: Alexander Stafeyev <astafeye>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 12.0 (Pike)CC: amuller, chrisw, ebarrera, ihrachys, nyechiel, ragiman, rcernin, srevivo, tfreger
Target Milestone: zstreamKeywords: Triaged, ZStream
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-neutron-2015.1.4-23.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1489066 Environment:
Last Closed: 2017-10-25 17:05:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1489066, 1489081    
Bug Blocks: 1489069, 1489070, 1489071, 1489072, 1504790, 1504791, 1505518, 1505520, 1505522, 1505524, 1505525, 1505526, 1505529    

Comment 3 Ihar Hrachyshka 2017-10-05 17:18:37 UTC
Reproduction steps would be:

- set up RH-OSP cloud (scenario doesn't really matter as long as it uses reference implementation)
- execute tempest suite against it a bunch of times
- check that neither of ovs/l3/dhcp agent logs have the error message "Another app is currently holding the xtables lock. Perhaps you want to use the -w option?" on any of the nodes

Comment 4 Toni Freger 2017-10-06 08:26:13 UTC
Seems like this fix didn't work since I do see those errors

[root@controller-0 neutron]# grep -iR "Another app is currently" /var/log/neutron/*
/var/log/neutron/dhcp-agent.log:Stderr: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
/var/log/neutron/l3-agent.log:Stderr: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

Comment 5 Toni Freger 2017-10-06 12:34:22 UTC
More information is added:
Setup 1 Controller and 1 computes


Stdin: # Generated by iptables-save v1.4.21 on Fri Oct  6 08:04:35 2017
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:neutron-dhcp-age-OUTPUT - [0:0]
:neutron-dhcp-age-PREROUTING - [0:0]
[0:0] -A PREROUTING -j neutron-dhcp-age-PREROUTING
[0:0] -A OUTPUT -j neutron-dhcp-age-OUTPUT
COMMIT
# Completed on Fri Oct  6 08:04:35 2017
# Generated by iptables-save v1.4.21 on Fri Oct  6 08:04:35 2017
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:neutron-dhcp-age-FORWARD - [0:0]
:neutron-dhcp-age-INPUT - [0:0]
:neutron-dhcp-age-OUTPUT - [0:0]
:neutron-dhcp-age-POSTROUTING - [0:0]
:neutron-dhcp-age-PREROUTING - [0:0]
:neutron-dhcp-age-mark - [0:0]
[0:0] -A PREROUTING -j neutron-dhcp-age-PREROUTING
[0:0] -A INPUT -j neutron-dhcp-age-INPUT
[0:0] -A FORWARD -j neutron-dhcp-age-FORWARD
[0:0] -A OUTPUT -j neutron-dhcp-age-OUTPUT
[0:0] -A POSTROUTING -j neutron-dhcp-age-POSTROUTING
[0:0] -A neutron-dhcp-age-PREROUTING -j neutron-dhcp-age-mark
[0:0] -A neutron-dhcp-age-POSTROUTING -p udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Fri Oct  6 08:04:35 2017
# Generated by iptables-save v1.4.21 on Fri Oct  6 08:04:35 2017
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]:OUTPUT ACCEPT [0:0]
:neutron-filter-top - [0:0]
:neutron-dhcp-age-FORWARD - [0:0]
:neutron-dhcp-age-INPUT - [0:0]
:neutron-dhcp-age-OUTPUT - [0:0]
:neutron-dhcp-age-local - [0:0]
[0:0] -A FORWARD -j neutron-filter-top
[0:0] -A OUTPUT -j neutron-filter-top
[0:0] -A neutron-filter-top -j neutron-dhcp-age-local
[0:0] -A INPUT -j neutron-dhcp-age-INPUT
[0:0] -A OUTPUT -j neutron-dhcp-age-OUTPUT
[0:0] -A FORWARD -j neutron-dhcp-age-FORWARD
COMMIT
# Completed on Fri Oct  6 08:04:35 2017

Stdout:
Stderr: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

2017-10-06 08:05:24.706 3816 ERROR neutron.agent.linux.utils [req-5a1f2d93-82e6-4d3b-a0f4-9ff26397557f ]
Command: ['ip', 'netns', 'exec', u'qdhcp-4f1ee8df-d696-471e-9576-b5388f52b45d', 'ip', '-4', 'route', 'replace', 'default', 'via', u'10.100.0.17', 'dev', 'tapdfa34054-97']

Comment 6 Ihar Hrachyshka 2017-10-09 18:09:08 UTC
Toni, please attach DHCP and L3 agent logs with the error so that we can inspect the issue. Note that the iptables command is first executed without -w parameter and then repeated with -w if the first call raised the error. Maybe the error is from the first attempt, and so it may be not a real issue.

Comment 7 Ihar Hrachyshka 2017-10-10 17:48:11 UTC
OK, I think I understand why it logs the error. This is because we don't pass log_as_error=False on first call to iptables. It's an cosmetic issue since the firewall is still set.

Comment 10 errata-xmlrpc 2017-10-25 17:05:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3069

Comment 11 Assaf Muller 2018-02-14 14:27:03 UTC
*** Bug 1545259 has been marked as a duplicate of this bug. ***