Bug 1489074 - iptables manager may fail to apply firewall rules if another iptables* process is being executed
Summary: iptables manager may fail to apply firewall rules if another iptables* proces...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: zstream
: 7.0 (Kilo)
Assignee: Ihar Hrachyshka
QA Contact: Alexander Stafeyev
URL:
Whiteboard:
: 1545259 (view as bug list)
Depends On: 1489066 1489081
Blocks: 1489069 1489070 1489071 1489072 1504790 1504791 1505518 1505520 1505522 1505524 1505525 1505526 1505529
TreeView+ depends on / blocked
 
Reported: 2017-09-06 16:05 UTC by Ihar Hrachyshka
Modified: 2022-08-16 12:38 UTC (History)
9 users (show)

Fixed In Version: openstack-neutron-2015.1.4-23.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1489066
Environment:
Last Closed: 2017-10-25 17:05:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1712185 0 None None None 2017-09-06 16:05:00 UTC
OpenStack gerrit 495974 0 None MERGED Make use of -w argument for iptables calls 2020-04-24 15:01:00 UTC
Red Hat Issue Tracker OSP-4690 0 None None None 2022-08-16 12:38:14 UTC
Red Hat Knowledge Base (Solution) 3204701 0 None None None 2017-10-03 12:50:09 UTC
Red Hat Product Errata RHBA-2017:3069 0 normal SHIPPED_LIVE openstack-neutron bug fix advisory 2017-10-25 21:02:55 UTC

Comment 3 Ihar Hrachyshka 2017-10-05 17:18:37 UTC
Reproduction steps would be:

- set up RH-OSP cloud (scenario doesn't really matter as long as it uses reference implementation)
- execute tempest suite against it a bunch of times
- check that neither of ovs/l3/dhcp agent logs have the error message "Another app is currently holding the xtables lock. Perhaps you want to use the -w option?" on any of the nodes

Comment 4 Toni Freger 2017-10-06 08:26:13 UTC
Seems like this fix didn't work since I do see those errors

[root@controller-0 neutron]# grep -iR "Another app is currently" /var/log/neutron/*
/var/log/neutron/dhcp-agent.log:Stderr: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
/var/log/neutron/l3-agent.log:Stderr: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

Comment 5 Toni Freger 2017-10-06 12:34:22 UTC
More information is added:
Setup 1 Controller and 1 computes


Stdin: # Generated by iptables-save v1.4.21 on Fri Oct  6 08:04:35 2017
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:neutron-dhcp-age-OUTPUT - [0:0]
:neutron-dhcp-age-PREROUTING - [0:0]
[0:0] -A PREROUTING -j neutron-dhcp-age-PREROUTING
[0:0] -A OUTPUT -j neutron-dhcp-age-OUTPUT
COMMIT
# Completed on Fri Oct  6 08:04:35 2017
# Generated by iptables-save v1.4.21 on Fri Oct  6 08:04:35 2017
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:neutron-dhcp-age-FORWARD - [0:0]
:neutron-dhcp-age-INPUT - [0:0]
:neutron-dhcp-age-OUTPUT - [0:0]
:neutron-dhcp-age-POSTROUTING - [0:0]
:neutron-dhcp-age-PREROUTING - [0:0]
:neutron-dhcp-age-mark - [0:0]
[0:0] -A PREROUTING -j neutron-dhcp-age-PREROUTING
[0:0] -A INPUT -j neutron-dhcp-age-INPUT
[0:0] -A FORWARD -j neutron-dhcp-age-FORWARD
[0:0] -A OUTPUT -j neutron-dhcp-age-OUTPUT
[0:0] -A POSTROUTING -j neutron-dhcp-age-POSTROUTING
[0:0] -A neutron-dhcp-age-PREROUTING -j neutron-dhcp-age-mark
[0:0] -A neutron-dhcp-age-POSTROUTING -p udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Fri Oct  6 08:04:35 2017
# Generated by iptables-save v1.4.21 on Fri Oct  6 08:04:35 2017
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]:OUTPUT ACCEPT [0:0]
:neutron-filter-top - [0:0]
:neutron-dhcp-age-FORWARD - [0:0]
:neutron-dhcp-age-INPUT - [0:0]
:neutron-dhcp-age-OUTPUT - [0:0]
:neutron-dhcp-age-local - [0:0]
[0:0] -A FORWARD -j neutron-filter-top
[0:0] -A OUTPUT -j neutron-filter-top
[0:0] -A neutron-filter-top -j neutron-dhcp-age-local
[0:0] -A INPUT -j neutron-dhcp-age-INPUT
[0:0] -A OUTPUT -j neutron-dhcp-age-OUTPUT
[0:0] -A FORWARD -j neutron-dhcp-age-FORWARD
COMMIT
# Completed on Fri Oct  6 08:04:35 2017

Stdout:
Stderr: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

2017-10-06 08:05:24.706 3816 ERROR neutron.agent.linux.utils [req-5a1f2d93-82e6-4d3b-a0f4-9ff26397557f ]
Command: ['ip', 'netns', 'exec', u'qdhcp-4f1ee8df-d696-471e-9576-b5388f52b45d', 'ip', '-4', 'route', 'replace', 'default', 'via', u'10.100.0.17', 'dev', 'tapdfa34054-97']

Comment 6 Ihar Hrachyshka 2017-10-09 18:09:08 UTC
Toni, please attach DHCP and L3 agent logs with the error so that we can inspect the issue. Note that the iptables command is first executed without -w parameter and then repeated with -w if the first call raised the error. Maybe the error is from the first attempt, and so it may be not a real issue.

Comment 7 Ihar Hrachyshka 2017-10-10 17:48:11 UTC
OK, I think I understand why it logs the error. This is because we don't pass log_as_error=False on first call to iptables. It's an cosmetic issue since the firewall is still set.

Comment 10 errata-xmlrpc 2017-10-25 17:05:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3069

Comment 11 Assaf Muller 2018-02-14 14:27:03 UTC
*** Bug 1545259 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.