Hide Forgot
Description of problem: I have ZONE=trusted in my ifcfg-br0 file. When I reboot, this is somehow changed to ZONE= which puts it in the default public zone which is breaking my VMs. Version-Release number of selected component (if applicable): firewalld-0.4.3.2-8.el7 NetworkManager-1.4.0-12.el7 systemd-219-30.el7 How reproducible: Every reboot Steps to Reproduce: 1. Have ZONE=trusted in ifcfg file 2. Reboot system 3. View ifcfg file Actual results: ZONE is blanked in ifcfg file Expected results: ZONE should be respected and kept in ifcfg file Additional info:
This appears to be a regression when compared to 7.2 which does not exhibit this problem. It also appears to be a function of having both NetworkManager and network active and enabled. In my scenario, I want NetworkManager available to control certain interfaces and need to use network because I have some openvswitch interfaces.
Is NM_CONTROLLED=no set in this ifcfg file?
NM_CONTROLLED has not existed in the files in 7.2 nor in 7.3. Since this is a Linux Bridge, does NM_CONTROLLED need to be set in the bridge ifcfg *and* the underlying physical interface? If it helps and doesn't break 7.2, I'd be happy to put it in.
NM_CONTROLLED=no tells NetworkManager not to use the ifcfg file. This is not explained in the Networking Guide, somehow. There are several things that might interfere with. There is a new firewalld version (0.4.3.2) and also a new NetworkManager version (1.4.0) in RHEL-7.3. Please check if the configuration of the bridge in NM shows the correct zone. I think the new NM version will try to take care about the bridge. Then it might either use another configuration (ifcfg or own) or the bridge is controlled by NM and the network service. This could then also result in issues. Can you try to set NM_CONTROLLED=no in the ifcfg file of that bridge to test of this is fixing your issue? Or use NM to control the bridge. I think this needs to be verified and therefore it would be good to get the sos information of that system.
Created attachment 1207196 [details] sosreport after initial install when it still works This is the sosreport after the initial install. Evidently network is the one that brought up the bridge: [root@foundation0 ~]# systemctl -l status network ● network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Active: active (exited) since Tue 2016-10-04 10:18:42 EDT; 6min ago Docs: man:systemd-sysv-generator(8) Process: 963 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=0/SUCCESS) Oct 04 10:18:42 foundation0.ilt.example.com systemd[1]: Starting LSB: Bring up/down networking... Oct 04 10:18:42 foundation0.ilt.example.com network[963]: Bringing up loopback interface: [ OK ] Oct 04 10:18:42 foundation0.ilt.example.com network[963]: Bringing up interface enp2s0: [ OK ] Oct 04 10:18:42 foundation0.ilt.example.com network[963]: Bringing up interface br0: [ OK ] Oct 04 10:18:42 foundation0.ilt.example.com systemd[1]: Started LSB: Bring up/down networking. [root@foundation0 ~]# firewall-cmd --get-active-zones external interfaces: eno1 trusted interfaces: br0 public interfaces: enp2s0 After I reboot, the ifcfg-br0 was moved to ifcfg-br0.old and the "new" ifcfg-br0 has the blank ZONE= [root@foundation0 ~]# firewall-cmd --get-active-zones public interfaces: br0 enp2s0 eno1
As an aside, what is responsible for creating the ifcfg-<iface>.old files in /etc/sysconfig/network-scripts/? That seems to be the culprit of pulling out the ZONE value.
Putting in NM_CONTROLLED=no has not helped. It really appears to be related to whatever process/service is creating those .old versions of the ifcfg- file. That process is not preserving the ZONE= value.
So, with NetworkManager disabled during the installation (as part of kickstart), something is still moving ifcfg-br0 to ifcfg-br0.old where "ZONE" is being "blanked".... So, something with "network.service" or "firewalld" is generating the .old file and failing to replicate the ZONE value.
Please try to apply this patch to firewall/core/fw_zone.py in /usr/lib/python2.7/site-packages/: --- firewall/core/fw_zone.py.old +++ firewall/core/fw_zone.py @@ -671,7 +671,7 @@ class FirewallZone(object): zone_transaction.add_post(self.__unregister_interface, _obj, interface_id) - zone_transaction.add_post(ifcfg_set_zone_of_interface, "", interface) + #zone_transaction.add_post(ifcfg_set_zone_of_interface, "", interface) if use_zone_transaction is None: zone_transaction.execute(True) This will stop the ifcfg file adaption in the remove_interface method on zones. With keeping the ifcfg file adaption in add_interface it should still be sufficient for people that are not using NetworkManager at all.
That patch has worked for the scenario where I have entirely disabled NetworkManager, am using just network.service, and still using firewalld for rules. This would work for me. I will test tomorrow with the original configuration of mish-mash NetworkManager and network.service both being active. Is there a timeline for incorporating this before release or am I looking at a post-release 7.3 rpm (perhaps not until 7.4?)
This is also working with the mishmash of NetworkManager and network.service both active. Though I have a separate bone to pick with network.service not respecting NM_CONTROLLED=yes of late.
Either I'm doing sth wrong or this is not working: [root@dhcp165 ~]# uname -a Linux dhcp165.coe.muc.redhat.com 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux [root@dhcp165 ~]# [root@dhcp165 ~]# diff -u /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py.old /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py --- /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py.old 2016-11-16 12:12:24.191360989 +0100 +++ /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py 2016-11-16 12:12:58.941175540 +0100 @@ -671,7 +671,7 @@ zone_transaction.add_post(self.__unregister_interface, _obj, interface_id) - zone_transaction.add_post(ifcfg_set_zone_of_interface, "", interface) + #zone_transaction.add_post(ifcfg_set_zone_of_interface, "", interface) if use_zone_transaction is None: zone_transaction.execute(True) [root@dhcp165 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0 [root@dhcp165 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 IPV6INIT=no BOOTPROTO=none ONBOOT=yes #ZONE=public DNS1=10.32.96.1 TYPE=Ethernet DEFROUTE=yes IPV4_FAILURE_FATAL=no NAME="System eth0" UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 IPADDR=10.32.111.165 PREFIX=20 GATEWAY=10.32.111.254 ZONE=test [root@dhcp165 ~]# nmcli con reload eth0 [root@dhcp165 ~]# nmcli con show 'System eth0' |grep -i zone connection.zone: test GENERAL.ZONE: test [root@dhcp165 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 IPV6INIT=no BOOTPROTO=none ONBOOT=yes #ZONE=public DNS1=10.32.96.1 TYPE=Ethernet DEFROUTE=yes IPV4_FAILURE_FATAL=no NAME="System eth0" UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 IPADDR=10.32.111.165 PREFIX=20 GATEWAY=10.32.111.254 ZONE=test [root@dhcp165 ~]# [root@dhcp165 ~]# firewall-cmd --zone=test --list-all test (active) target: default icmp-block-inversion: no interfaces: eth0 sources: services: ssh ports: protocols: masquerade: no forward-ports: sourceports: icmp-blocks: rich rules: [root@dhcp165 ~]# [root@dhcp165 ~]# systemctl restart network [root@dhcp165 ~]# [root@dhcp165 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 IPV6INIT=no BOOTPROTO=none ONBOOT=yes #ZONE=public DNS1=10.32.96.1 TYPE=Ethernet DEFROUTE=yes IPV4_FAILURE_FATAL=no NAME="System eth0" UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 IPADDR=10.32.111.165 PREFIX=20 GATEWAY=10.32.111.254 [root@dhcp165 ~]# nmcli con show 'System eth0' |grep -i zone connection.zone: -- GENERAL.ZONE: -- [root@dhcp165 ~]# [root@dhcp165 ~]# firewall-cmd --zone=test --list-all test target: default icmp-block-inversion: no interfaces: sources: services: ssh ports: protocols: masquerade: no forward-ports: sourceports: icmp-blocks: rich rules: [root@dhcp165 ~]# [root@dhcp165 ~]# NetworkManager and network are both enabled and even if the fix is applied (if I'm not mistaken) after a network restart or reboot of the system the configuration is gone which makes my customers network unusable and we need a fix here asap
btw, when doing just an ifdown eth0 the ZONE=test becomes ZONE=
Further testing: playing around I found, if I disable NetworkManager on above system, add the ZONE=test entry again, reboot, enable NetworkManager and reboot again, it still holds the ZONE=test configuration on a System I disabled NetworkManager pre upgrade to 7.3 there are no problems and the ZONE entry in ifcfg file is always valid and the interface stays in the defined zone. If enabling NetworkManager after the update this does not influence the ZONE entry in ifcfg file and so the interface keeps in the firewall zone it should be.
ok, I cannot find any method to set a zone for an interface in 7.3 using NetworkManager, neither via ifcfg (ZONE= ) and 'firewall-cmd --permanent --zone=test --add-interface=eth0' which aren't supported anyways but also not via nmcli c modify 'System eth0' connection.zone test as this is reset after reboot... and also not with the fix from comment #11 [root@dhcp165 ~]# [root@dhcp165 ~]# diff -u /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py.old --- /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py 2016-11-16 15:13:21.111542507 +0100 +++ /usr/lib/python2.7/site-packages/firewall/core/fw_zone.py.old 2016-11-16 15:12:56.851717795 +0100 @@ -671,7 +671,7 @@ zone_transaction.add_post(self.__unregister_interface, _obj, interface_id) -# zone_transaction.add_post(ifcfg_set_zone_of_interface, "", interface) + zone_transaction.add_post(ifcfg_set_zone_of_interface, "", interface) if use_zone_transaction is None: zone_transaction.execute(True) [root@dhcp165 ~]# nmcli con show 'System eth0' |grep -i zone connection.zone: -- GENERAL.ZONE: -- [root@dhcp165 ~]# nmcli c modify 'System eth0' connection.zone test [root@dhcp165 ~]# [root@dhcp165 ~]# nmcli con show 'System eth0' |grep -i zone connection.zone: test GENERAL.ZONE: test [root@dhcp165 ~]# [root@dhcp165 ~]# reboot $ ssh root.muc.redhat.com Last login: Wed Nov 16 15:10:28 2016 from 10.32.64.93 [root@dhcp165 ~]# [root@dhcp165 ~]# nmcli con show 'System eth0' |grep -i zone connection.zone: -- GENERAL.ZONE: -- [root@dhcp165 ~]# and the same happens when I just do a ifdown eth0. the only ways I got this working is either: - disable network.service (after the upgrade I nevertheless had to set the zone once again via nmcli c modify 'System eth0' connection.zone test to make it persistent again) or -disable NetworkManager (After that the ZONE=.. in ifcfg file worked in RHEL7.2 and after upgrading to 7.3 w/o any further issues) pre or after the upgrade. As this worked in RHEL7.2 I consider this a regression, or do I miss sth here ?
Created attachment 1221174 [details] protocoll of tests and issues
The ifcfg .old file is generated by firewalld. This has been fixed upstream with https://github.com/t-woerner/firewalld/commit/fe6cf16e5a5ef3e49cdb554af8cf18024371554a
@daniel With the firewalld version in 7.3 the zone setting for connections is not saved in firewalld as long as NetworkManager is under control of the interface. This applies to firewall-cmd .. --change-interface= and the GUIs. This is done to only have one place where the zone setting for this connection/interface is stored. Please add the patch from comment 11 after doing the update. The additional assignment of the patch in comment 20 is making sure that an ifcfg backup save file is using the .bak extension and not the .old extension. ifcfg files with the .bak extension are ignored by NetworkManager and the network service. With these two patches I do not see any issues any more after reboots and ifdowns/ifups and network service restarts. Please verify if this is also the case for you. If needed I will create a test package with these two patches applied.
Created attachment 1221612 [details] test proposed fixes
Thomas, find attached my test of the suggested fixes attached as shown in previous update. Test plan was: Steps: RHEL7.2: 1) configure with interface in zone trust : # systemctl list-units --all|egrep -i " network\.|NetworkManager" # cat /etc/sysconfig/network-scripts/ifcfg-eth0 # firewall-cmd --zone=public --list-all # firewall-cmd --zone=test --list-all # nmcli con show 'System eth0' |grep -i zone ------- # nmcli c modify 'System eth0' connection.zone test # nmcli con show 'System eth0' |grep -i zone # cat /etc/sysconfig/network-scripts/ifcfg-eth0 # firewall-cmd --zone=test --list-all 2)reboot--- # nmcli con show 'System eth0' |grep -i zone # cat /etc/sysconfig/network-scripts/ifcfg-eth0 # firewall-cmd --zone=test --list-all --> expected: configuration persistent ==> OK 3) update to RHEL7.3 and reboot # nmcli con show 'System eth0' |grep -i zone # cat /etc/sysconfig/network-scripts/ifcfg-eth0 # firewall-cmd --zone=test --list-all --> expected: configuration gone ==> OK 4) implement fixes, reconfigure ==> done # nmcli c modify 'System eth0' connection.zone test # nmcli con show 'System eth0' |grep -i zone # cat /etc/sysconfig/network-scripts/ifcfg-eth0 # firewall-cmd --zone=test --list-all 5) reboot # nmcli con show 'System eth0' |grep -i zone # cat /etc/sysconfig/network-scripts/ifcfg-eth0 # firewall-cmd --zone=test --list-all --> expected: configuration persistant ==> OK -> second reboot,expected ok ==> OK 6) second test 6.1 revert to buggy scripts in RHEL7.3 and reboot --> expected is to have issue again ==> OK 6.2 put back in fixes and edit just ifcfg file, reboot (will this work w/o nmcli con reload?) --> expected, will do ==>OK So I'd say your fix is working indeed, but the renaming did not work for me, see attached protocol, perhaps I made a mistake. Cheers, Daniel
(In reply to daniel from comment #23) > Thomas, > ... > > So I'd say your fix is working indeed, but the renaming did not work for me, > see attached protocol, perhaps I made a mistake. > > Cheers, > Daniel The renaming is working, there is no ifcfg-X.old in use even after changing the ifcfg file. Or are you talking about another rename?
Here is the upstream patch: https://github.com/t-woerner/firewalld/commit/636e01137515f3830c655619096e9642651a674c
I can confirm that the solution offered in Comment 11 worked on three servers affected at my site so far. One server had NO existing ZONE=, the two others both had ZONE= specified for each network interface. In my environment this only seems to be happening to systems that are using a SCAP baseline. Could those others of you who have had this occur check if you are utilizing a SCAP baseline as well? Thank you, Justin
From what I discovered this bug affects ifcfg's matched by DEVICE but not by HWADDR. I fixed it by disabling the network service as I use NetworkManager anyway.
*** Bug 1385521 has been marked as a duplicate of this bug. ***
This bug was mistakenly closed, it's in the previous state now.
I got the following workaround without using the patch, not sure how reliable it is and not sure how it really works, but thought I leave it here in case it uncovers weird problems. For example I got eth1 in public zone and want to put it into trusted zone. 1) ifdown eth1 2) Add ZONE=trusted to ifcfg-eth1. 3) ifup eth1 4) firewall-cmd --zone=trusted --list-all --> eth1 is now in trusted zone. 5) reboot 6) firewall-cmd --zone=trusted --list-all --> eth1 still in trusted zone. If I do 2) then 1) then ifdown eth1 wipes ZONE in ifcfg-eth1. So ifdown is not called when the server is rebooted? NetworkManager is disabled and not running, network.service is enabled and running. So as a result I got something like this in puppet to manage it: augeas { "${interface}_ZONE": context => "/files/etc/sysconfig/network-scripts/ifcfg-${interface}", changes => 'set ZONE trusted', }~> exec { "profile__network_stop_${interface}": command => "ifdown ${interface}", path => '/usr/sbin', refreshonly => true, }-> augeas { "${interface}_ZONE_again": context => "/files/etc/sysconfig/network-scripts/ifcfg-${interface}", changes => 'set ZONE trusted', }~> exec { "profile__network_start_${interface}": command => "ifup ${interface}", path => '/usr/sbin', refreshonly => true, } Not ideal but works.
Here's a simple workaround based on the fact that both firewalld and ifdown (/usr/sbin/usernetctl) both save the old configuration when they write a new configuration: [root@beaver-ocata network-scripts]# mv /usr/sbin/ifdown /usr/sbin/ifdown-original [root@beaver-ocata network-scripts]# vi /usr/sbin/ifdown #!/bin/bash /usr/sbin/ifdown-original $1 $2 if [ -f /etc/sysconfig/network-scripts/ifcfg-${1}.old ]; then mv /etc/sysconfig/network-scripts/ifcfg-${1}.old /etc/sysconfig/network-scripts/ifcfg-${1} fi
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:1934
From https://github.com/moby/moby/issues/16137#issuecomment-394630145, I can confirm this issue is not fixed yet, in some environment. I can see at least 3 projects (mine, Angelinsky7 and PMarci) where our CentOs/RedHat is >= 7.3 and this still happen.