Created attachment 1689263 [details] Log from systemctl status firewalld Description of problem: After updating to F32 with a relatively modest set of firewall rules, firewalld appears to start up only partially, and fails to load the firewall configuration completely. Version-Release number of selected component (if applicable): firewalld-0.8.2-3.fc32.noarch How reproducible: Always Steps to Reproduce: 1. Update F31 to F32 2. 3. Actual results: Everything looks normal in firewall-config, but: # firewall-cmd --list-interfaces [ no output ] # firewall-cmd --get-active-zone [ no output ] In firewall-config reloading firewalld pops up with a Python exception. The same exception is logged in systemctl, see attached. Expected results: firewall starts normally Additional info: See attachment
This looks like nftables compatibility issue. Switching the backend back to iptables restored existing functionality.
Created attachment 1689265 [details] A few more errors found in /var/log Found a few more errors in /var/log/firewalld
Created attachment 1689266 [details] FedoraServer.xml with IP addresses masked out, referenced ipsets are type="hash:net"
Created attachment 1689267 [details] /etc/firewalld/direct.xml (the referenced ipset is type="hash:net"
/var/log/firewalld shows firewalld failed to delete its own nftables tables. That can occur of some other entity (e.g. nftables service) causes a flush of nftables rules (i.e. 'nft flush ruleset').
It seems this issue was reported to RHEL 8.2 here, and now QA status. https://bugzilla.redhat.com/show_bug.cgi?id=1817205 I hope the patch will be applied to Fedora 32 too.
(In reply to Jun Aruga from comment #6) > It seems this issue was reported to RHEL 8.2 here, and now QA status. > https://bugzilla.redhat.com/show_bug.cgi?id=1817205 > > I hope the patch will be applied to Fedora 32 too. They are separate issues. The Conflicts already exists in Fedora 32. # grep Conflicts /usr/lib/systemd/system/firewalld.service Conflicts=iptables.service ip6tables.service ebtables.service ipset.service nftables.service ^^^^^^^^^^^^^^^^ # dnf info firewalld Installed Packages Name : firewalld Version : 0.8.3 Release : 1.fc32
@Sam, you originally reported this against firewalld-0.8.2-3.fc32.noarch. Your logs indicate that it was possibly caused by the simultaneous use of both services: firewalld, and nftables. This was addressed in firewalld-0.8.3-1. Please check that it's still an issue. If you don't reply I'll assume this is already fixed.
This issue still exists as of firewalld-0.8.3-1.fc32.noarch [root@shorty tmp]# rpm -q firewalld firewalld-0.8.3-1.fc32.noarch Aug 29 10:03:56 shorty.email-scan.com systemd[1]: Starting firewalld - dynamic firewall daemon... Aug 29 10:03:57 shorty.email-scan.com systemd[1]: Started firewalld - dynamic firewall daemon. Aug 29 10:03:57 shorty.email-scan.com firewalld[3394]: ERROR: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: File exists internal:0:0-0: Error: Could not process rule: File exists internal:0:0-0: Error: Could not process rule: File exists This seems like to be some sort of incompatibility caused by some nftables-incompatible rule that was originally created before nftables became the default backend. Setting FirewallBackend=iptables in firewalld-server.conf makes firewalld start successfully. It's most likely a single direct rule that I have installed. I haven't had the opportunity to test this theory: <?xml version="1.0" encoding="utf-8"?> <direct> <rule ipv="ipv4" table="raw" chain="PREROUTING" priority="0">-m set --match-set dropips src -j DROP</rule> </direct> Well, if this means that firewalld can't install this rule, then that's that; but this ends up with firewalld loading an incomplete set of regular rules, altogether. And this was a routine upgrade, firewalld-server.xml was not customized, none of the xml files were manually customized, prior to the upgrade, all configuration was assembled, over time, via firewall-config, and the upgrade ended up getting busted, until I figured out to manually set FirewallBackend=iptables
The direct rule is fine. It should still work with the nftables backend. Can you set IndividualCalls=yes in /etc/firewalld/firewalld.conf and add --debug option in /etc/sysconfig/firewalld? This will help point to the issue.
Sep 04 07:05:39 shorty.email-scan.com firewalld[73547]: ERROR: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: interval overlaps with an existing one internal:0:0-0: Error: Could not process rule: File exists JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"element": {"family": "inet", "table": "firewalld", "name": "chinanet", "elem": [{"prefix": {"addr": "183.128.0.0", "len": 11}}]}}}, {"add": {"element": {"family": "ip", "table": "firewalld", "name": "chinanet", "elem": [{"prefix": {"addr": "183.128.0.0", "len": 11}}]}}}, {"add": {"element": {"family": "ip6", "table": "firewalld", "name": "chinanet", "elem": [{"prefix": {"addr": "183.128.0.0", "len": 11}}]}}}]} I had an ipset with a couple of overlapping IP address ranges. This was accepted with the iptables backend, but rejected by the nftables backend.
(In reply to Sam Varshavchik from comment #11) > Sep 04 07:05:39 shorty.email-scan.com firewalld[73547]: ERROR: > COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: interval > overlaps with an existing one > > internal:0:0-0: > Error: Could not process rule: File exists > > > JSON blob: > {"nftables": > [{"metainfo": {"json_schema_version": 1}}, {"add": {"element": {"family": > "inet", "table": "firewalld", "name": "chinanet", "elem": [{"prefix": > {"addr": "183.128.0.0", "len": 11}}]}}}, {"add": {"element": {"family": > "ip", "table": "firewalld", "name": "chinanet", "elem": [{"prefix": {"addr": > "183.128.0.0", "len": 11}}]}}}, {"add": {"element": {"family": "ip6", > "table": "firewalld", "name": "chinanet", "elem": [{"prefix": {"addr": > "183.128.0.0", "len": 11}}]}}}]} > > I had an ipset with a couple of overlapping IP address ranges. > > This was accepted with the iptables backend, but rejected by the nftables > backend. Thanks. This made it clear whats going on: 1. firewalld needs to use the "auto-merge" feature of sets to a allow element coalescing. 2. nftables needs various upstream fixes (kernel) to fix some set element overlap detection and coalescing issues. I'll keep this bug open for item #1, but item #2 will come via a kernel update.
Don't know if this is related. Started seeing on Aug 27. Only system change would be auto updates: Sep 05 12:06:56 mail.yorknation.com firewalld[625]: ERROR: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "raw_PREROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "raw_PRE_public"}}]}}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "mangle_PREROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "mangle_PRE_public"}}]}}}, {"insert": {"rule": {"family": "ip", "table": "firewalld", "chain": "nat_PREROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "nat_PRE_public"}}]}}}, {"insert": {"rule": {"family": "ip6", "table": "firewalld", "chain": "nat_PREROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "nat_PRE_public"}}]}}}, {"insert": {"rule": {"family": "ip", "table": "firewalld", "chain": "nat_POSTROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "oifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "nat_POST_public"}}]}}}, {"insert": {"rule": {"family": "ip6", "table": "firewalld", "chain": "nat_POSTROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "oifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "nat_POST_public"}}]}}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_INPUT_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "filter_IN_public"}}]}}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_FORWARD_IN_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "filter_FWDI_public"}}]}}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_FORWARD_OUT_ZONES", "expr": [{"match": {"left": {"meta": {"key": "oifname"}}, "op": "==", "right": "enp3s0"}}, {"goto": {"target": "filter_FWDO_public"}}]}}}]}
Clearing the needinfo flag that bugzilla's been bugging me about, which I forgot to do in comment 11.
Is there any kind of workaround for this? It prevents starting VMs on Fedora: $ sudo virsh start freebsd error: Failed to start domain freebsd error: Requested operation is not valid: network 'default' is not active $ sudo virsh net-start default error: Failed to start network default error: error from service: GDBus.Error:org.fedoraproject.FirewallD1.Exception: COMMAND_FAILED: 'python-nftables' failed: JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_INPUT_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "virbr0"}}, {"goto": {"target": "filter_IN_libvirt"}}]}}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_FORWARD_OUT_ZONES", "expr": [{"match": {"left": {"meta": {"key": "oifname"}}, "op": "==", "right": "virbr0"}}, {"goto": {"target": "filter_FWDO_libvirt"}}]}}}, {"insert": {"rule": {"family": "ip", "table": "firewalld", "chain": "nat_POSTROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "oifname"}}, "op": "==", "right": "virbr0"}}, {"goto": {"target": "nat_POST_libvirt"}}]}}}, {"insert": {"rule": {"family": "ip6", "table": "firewalld", "chain": "nat_POSTROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "oifname"}}, "op": "==" $ sudo firewall-cmd --reload Error: COMMAND_FAILED: 'python-nftables' failed: JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"table": {"family": "inet", "name": "firewalld_policy_drop"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld_policy_drop", "name": "filter_input", "type": "filter", "hook": "input", "prio": 9, "policy": "drop"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld_policy_drop", "name": "filter_forward", "type": "filter", "hook": "forward", "prio": 9, "policy": "drop"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld_policy_drop", "name": "filter_output", "type": "filter", "hook": "output", "prio": 9, "policy": "drop"}}}, {"add": {"rule": {"family": "inet", "table": "firewalld_policy_drop", "chain": "filter_input", "expr": [{"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["established", "related"]}}}, {"accept": null}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld_policy_drop", "chain": "filter_forward", "expr": [{"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["established", "related"]}}}, {"accept": null}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld_policy_drop", "chain": "filter_output", "expr": [{"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["established", "related"]}}}, {"accept": null}]}}}]} $ rpm -q firewalld firewalld-0.9.1-1.fc34.noarch
You need to sift through your ipset, find overlapping CIDRs, and remove the overlapping ranges manually. Once I did that, the nftables backend became happy.
(In reply to Richard W.M. Jones from comment #15) > Is there any kind of workaround for this? It prevents starting VMs on It looks like you're hitting a different issue. This bug is specific to overlapping ipsets. You don't appear to be using them. > $ rpm -q firewalld > firewalld-0.9.1-1.fc34.noarch This bug is filed against f32, not rawhide. Please file a bug against rawhide and inclued `/var/log/firewalld`.
This message is a reminder that Fedora 32 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '32'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 32 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This is still an issue after upgrade to Fedora 34. All my libvirt environment is unusable. Error starting network 'redhat-net': error from service: GDBus.Error:org.fedoraproject.FirewallD1.Exception: COMMAND_FAILED: 'python-nftables' failed: JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_pre"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_log"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_deny"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_allow"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_post"}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt", "expr": [{"jump": {"target": "filter_IN_libvirt_pre"}}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt", "expr": [{"jump": {"target": "filter_IN_libvirt_log"}} Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 65, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 101, in tmpcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn ret = fn(self, *args, **kwargs) File "/usr/share/virt-manager/virtManager/object/network.py", line 69, in start self._backend.create() File "/usr/lib64/python3.9/site-packages/libvirt.py", line 3436, in create raise libvirtError('virNetworkCreate() failed') libvirt.libvirtError: error from service: GDBus.Error:org.fedoraproject.FirewallD1.Exception: COMMAND_FAILED: 'python-nftables' failed: JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_pre"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_log"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_deny"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_allow"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_libvirt_post"}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt", "expr": [{"jump": {"target": "filter_IN_libvirt_pre"}}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt", "expr": [{"jump": {"target": "filter_IN_libvirt_log"}} ----------------------------- I tried to change from nftables to iptables in the /etc/firewalld/firewalld-workstation.conf and then I have an issue loading the zone file and restoring iptables: ERROR: Failed to load zone file '/etc/firewalld/zones/FedoraWorkstation.xml': ALREADY_ENABLED: '8080:tcp' already in 'ERROR: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore: line 129 failed ERROR: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 144 failed ERROR: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 144 failed ERROR: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore: line 52 failed ERROR: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 52 failed ERROR: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 52 failed
same problem on F34: root@fedora]# virsh net-start default error: Failed to start network default error: error from service: GDBus.Error:org.fedoraproject.FirewallD1.Exception: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory internal:0:0-0: Error: Could not process rule: No such file or directory JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"insert": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_INPUT_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "virbr0"}}, {"goto": {"target": "filter_IN_libvirt"}}]}}}, {"insert": {"rule": {"family": "inet", [root@fedora]# working after switching to iptables from nftables in /etc/firewalld/firewalld-workstation.conf
@Richard, @Andre, you two are seeing a different issue. Your issue is related to invalid configuration. See bug 1914935 comment 15. Please comment over there to avoid muddying this issue with unrelated information. This issue is caused by overlapping ipset entries as identified in comment 12.
Hi, I've similar error with podman, just trying to run the stock redis container (docker.io/library/redis:latest) with podman and it fails with error: ERRO[0000] Error adding network: failed to add the address 10.88.0.2/32 to trusted zone: COMMAND_FAILED: 'python-nftables' failed: JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_trusted"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_trusted_pre"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_trusted_log"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_trusted_deny"}}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_trusted_all (......) it started to appear after upgrade from F33 to F34.
I've found a tool which aggregates CIDR addresses into non-overlapping ranges. This seems to be a workaround for the present Firewalld bug. The tool is https://github.com/job/aggregate6
This message is a reminder that Fedora Linux 34 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '34'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 34 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
Fedora Linux 34 entered end-of-life (EOL) status on 2022-06-07. Fedora Linux 34 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. Thank you for reporting this bug and we are sorry it could not be fixed.