Bug 1369180

Summary: Sometimes firewalld is not flushing the firewall rules on shutdown
Product: Red Hat Enterprise Linux 7 Reporter: Fabian Deutsch <fdeutsch>
Component: firewalldAssignee: Eric Garver <egarver>
Status: CLOSED WONTFIX QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: medium    
Version: 7.2CC: atragler, danken, fdeutsch, lveyde, pvrabec, rkhan, sukulkar, todoleza
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-10 14:50:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Fabian Deutsch 2016-08-22 15:19:59 UTC
Description of problem:
Sometimes when firewalld is shut down, the rules are not getting removed completely

Version-Release number of selected component (if applicable):
RHEL 7.2 all updates

How reproducible:
Sometimes ~10%

Steps to Reproduce:
1. Stop firewalld
2. iptables -L
3.

Actual results:
Sometimes some rules remain

Expected results:
No rules remain at any time

Additional info:

Comment 1 Thomas Woerner 2016-08-22 15:24:46 UTC
What kind of rules are left over? Please add examples.

Comment 2 Fabian Deutsch 2016-08-22 15:43:54 UTC
The default rules are sometimes left.

Lev will probably paste the rules once he encounters the bug again.

Comment 3 Lev Veyde 2016-08-23 10:45:09 UTC
Chain FWDO_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain FWDO_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain FWDO_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT_ZONES (1 references)
 pkts bytes target     prot opt in     out     source               destination
   28  2236 IN_public  all  --  ens3   *       0.0.0.0/0            0.0.0.0/0           [goto]
    0     0 IN_public  all  --  +      *       0.0.0.0/0            0.0.0.0/0           [goto]

Chain INPUT_ZONES_SOURCE (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain IN_public (2 references)
 pkts bytes target     prot opt in     out     source               destination
   28  2236 IN_public_log  all  --  *      *       0.0.0.0/0            0.0.0.0/0
   28  2236 IN_public_deny  all  --  *      *       0.0.0.0/0            0.0.0.0/0
   28  2236 IN_public_allow  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain IN_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination
   10   600 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22 ctstate NEW
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            224.0.0.251          udp dpt:5353 ctstate NEW

Chain IN_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain IN_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination
[root@lago_phase_1_suite_storage ~]# 
    0     0 FWDO_public_allow  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FWDO_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain FWDO_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain FWDO_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT_ZONES (1 references)
 pkts bytes target     prot opt in     out     source               destination
   28  2236 IN_public  all  --  ens3   *       0.0.0.0/0            0.0.0.0/0           [goto]
    0     0 IN_public  all  --  +      *       0.0.0.0/0            0.0.0.0/0           [goto]

Chain INPUT_ZONES_SOURCE (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain IN_public (2 references)
 pkts bytes target     prot opt in     out     source               destination
   28  2236 IN_public_log  all  --  *      *       0.0.0.0/0            0.0.0.0/0
   28  2236 IN_public_deny  all  --  *      *       0.0.0.0/0            0.0.0.0/0
   28  2236 IN_public_allow  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain IN_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination
   10   600 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22 ctstate NEW
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            224.0.0.251          udp dpt:5353 ctstate NEW

Chain IN_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain IN_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination



# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

Aug 23 06:08:24 lago_phase_1_suite_storage systemd[1]: Starting firewalld - dynamic firewall daemon...
Aug 23 06:08:24 lago_phase_1_suite_storage systemd[1]: Started firewalld - dynamic firewall daemon.
Aug 23 06:09:22 lago_phase_1_suite_storage systemd[1]: Stopping firewalld - dynamic firewall daemon...
Aug 23 06:09:22 lago_phase_1_suite_storage systemd[1]: Stopped firewalld - dynamic firewall daemon.

Comment 4 Thomas Woerner 2016-08-23 11:11:24 UTC
Are there any errors in the log? The CleanupOnExit setting in firewalld.conf is still set to yes?

Comment 5 Thomas Woerner 2016-08-23 11:14:08 UTC
If there are no errors and if CleanupOnExit is still set to yes, then please enable the debug mode in firewalld by setting "FIREWALLD_ARGS=--debug" in /etc/sysconfig/firewalld and attach the file /var/log/firewalld.log to this bug after starting and stopping firewalld and still having the left over rules.

Comment 6 Lev Veyde 2016-08-24 12:52:32 UTC
(In reply to Thomas Woerner from comment #5)
> If there are no errors and if CleanupOnExit is still set to yes, then please
> enable the debug mode in firewalld by setting "FIREWALLD_ARGS=--debug" in
> /etc/sysconfig/firewalld and attach the file /var/log/firewalld.log to this
> bug after starting and stopping firewalld and still having the left over
> rules.

CleanupOnExit is set to yes, that is the default value.

I modified the /etc/sysconfig/firewalld as asked and the only thing in the /var/log/firewalld is:

2016-08-24 08:11:54 WARNING: FedoraServer: INVALID_SERVICE: cockpit

Comment 7 Thomas Woerner 2016-09-02 13:11:46 UTC
This does not explain any left overs. It there would be an issue while stopping firewalld, then it would be reported. Do you see this also with the firewalld version in 7.3 beta?

Could it be that something is accessing the firewalld D-Bus interface after it was stopped by systemd?

This happended in the past sometimes if the dependency to the firewalld service was not there or not correct.

Do you see a firewalld start before the system goes down?

Comment 8 Lev Veyde 2016-09-13 08:19:40 UTC
(In reply to Thomas Woerner from comment #7)
> This does not explain any left overs. It there would be an issue while
> stopping firewalld, then it would be reported. Do you see this also with the
> firewalld version in 7.3 beta?
> 
> Could it be that something is accessing the firewalld D-Bus interface after
> it was stopped by systemd?
> 
> This happended in the past sometimes if the dependency to the firewalld
> service was not there or not correct.
> 
> Do you see a firewalld start before the system goes down?

Haven't tested it with RHEL 7.3 beta.

No idea if anything accesses the firewalld through D-Bus, however as we caught this in one of the Lago env., with the exactly same OS images being used for each run, and only in some cases we see the firewall issue, I doubt if that is the configuration issue.

As we already pointed out this seemed to happen consistently, but not every time we created the env.

Comment 9 Dan Kenigsberg 2018-02-10 14:50:24 UTC
Lev, do we still see this? Let us reopen with fresh data if this still bothers us.

Comment 10 Lev Veyde 2018-02-19 15:12:39 UTC
(In reply to Dan Kenigsberg from comment #9)
> Lev, do we still see this? Let us reopen with fresh data if this still
> bothers us.

Haven't tested this recently, will re-open if it will re-occur.