Bug 2362648 - The newest iptables upraade breaks docker
Summary: The newest iptables upraade breaks docker
Keywords:
Status: MODIFIED
Alias: None
Product: Fedora
Classification: Fedora
Component: iptables
Version: 42
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2025-04-28 06:10 UTC by Dominik Zogg
Modified: 2026-05-06 12:46 UTC (History)
23 users (show)

Fixed In Version: iptables-1.8.11-7.fc42
Clone Of:
Environment:
Last Closed: 2025-04-29 20:40:22 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 2338712 0 unspecified CLOSED Latest rawhide python-firewall has wrong paths to iptables/ip6tables 2025-04-28 11:50:40 UTC

Description Dominik Zogg 2025-04-28 06:10:59 UTC
Description of problem:

I did an upgrade this morning and it broke docker (Service cannot be started). In the logs (journal) it mentioned iptables. So i searched for a iptables package that get ugraded, which was the case for:

- iptables-libs
- iptables-nft

I do not know which of both (or both) are to blame, but downgrading fixed it.

Downgrading:
 iptables-libs                                                                                                x86_64                     1.8.11-4.fc42                                                                                                 fedora                                                           1.8 MiB
   ersetze iptables-libs                                                                                      x86_64                     1.8.11-6.fc42                                                                                                 updates                                                          1.5 MiB
 iptables-nft                                                                                                 x86_64                     1.8.11-4.fc42                                                                                                 fedora                                                         537.5 KiB
   ersetze iptables-nft                                                                                       x86_64                     1.8.11-6.fc42                                                                                                 updates                                                        465.6 KiB


Version-Release number of selected component (if applicable):

iptables-nft.11-6.fc42
iptables-libs.11-6.fc42


How reproducible:

Use 1.8.11-4.fc42  and it works upgrade to 1.8.11-6.fc42 and it doesn't.
I use docker, as its provided by Fedora, not from the Docker website.


Actual results:

Docker service cannot be started.


Expected results:

Docker service can be started.

Comment 1 Dominik Zogg 2025-04-28 06:14:05 UTC
I set it on high, cause if someone uses docker on Fedora as a server (not like me as a deveoper machine) they will have a bad day.

Comment 2 Martin 2025-04-28 08:34:25 UTC
What is the actual problem with iptables-nft?

The version 1.8.11-4 introduced with Fedora 42 breaks Dockers custom networks. It was fixed with 1.8.11-5.
See: https://github.com/docker/for-linux/issues/1525#issuecomment-2815436491

Now the version 1.8.11-6 breaks Docker again completely, because Docker can't find iptables.

Who is responsible for this horror?

Comment 3 Dominik Zogg 2025-04-28 08:50:44 UTC
Hi @comp 

I use docker provided by Fedora itself, you use the packages from docker themself?

Cause 1.8.11-4 works with the onces provided by Fedora. It may break one fixing the other.

This are the relavant docker pages and direct deps, i got:

```
sudo dnf remove docker-cli
Paket                                               Architektur       Version                                              Paketquelle                    Größe
Wird entfernt:
 docker-cli                                         x86_64            27.5.1-1.fc42                                        fedora                      27.9 MiB
Abhängige Pakete werden entfernt:
 docker-buildx                                      x86_64            0.20.1-2.fc42                                        updates                     65.6 MiB
 docker-compose                                     x86_64            2.33.1-1.fc42                                        fedora                      68.1 MiB
 moby-engine                                        x86_64            27.5.1-1.fc42                                        fedora                     105.2 MiB
Nicht benötigte Abhängigkeiten werden entfernt:
 docker-compose-switch                              x86_64            1.0.5-2.fc42                                         fedora                       3.4 MiB
 moby-engine-nano                                   noarch            27.5.1-1.fc42                                        fedora                     102.1 KiB
 moby-filesystem                                    noarch            27.5.1-1.fc42                                        fedora                       0.0   B
 tini-static                                        x86_64            0.19.0-10.fc42                                       fedora                     773.9 KiB
```

Comment 4 Martin 2025-04-28 08:57:12 UTC
(In reply to Dominik Zogg from comment #3)
> Hi @comp 
> 
> I use docker provided by Fedora itself, you use the packages from docker
> themself?
> 
> Cause 1.8.11-4 works with the onces provided by Fedora. It may break one
> fixing the other.
> 
> This are the relavant docker pages and direct deps, i got:
> 
> ```
> sudo dnf remove docker-cli
> Paket                                               Architektur      
> Version                                              Paketquelle            
> Größe
> Wird entfernt:
>  docker-cli                                         x86_64           
> 27.5.1-1.fc42                                        fedora                 
> 27.9 MiB
> Abhängige Pakete werden entfernt:
>  docker-buildx                                      x86_64           
> 0.20.1-2.fc42                                        updates                
> 65.6 MiB
>  docker-compose                                     x86_64           
> 2.33.1-1.fc42                                        fedora                 
> 68.1 MiB
>  moby-engine                                        x86_64           
> 27.5.1-1.fc42                                        fedora                 
> 105.2 MiB
> Nicht benötigte Abhängigkeiten werden entfernt:
>  docker-compose-switch                              x86_64           
> 1.0.5-2.fc42                                         fedora                 
> 3.4 MiB
>  moby-engine-nano                                   noarch           
> 27.5.1-1.fc42                                        fedora                 
> 102.1 KiB
>  moby-filesystem                                    noarch           
> 27.5.1-1.fc42                                        fedora                 
> 0.0   B
>  tini-static                                        x86_64           
> 0.19.0-10.fc42                                       fedora                 
> 773.9 KiB
> ```

Yes, I use the packages from Docker, not from Fedora. And I want to stick with the "original" packages from Docker.
docker-ce-3:28.1.1-1.fc42

Comment 5 Martin 2025-04-28 09:13:46 UTC
I just downloaded iptables-nft version 1.8.11-5 from https://koji.fedoraproject.org/koji/buildinfo?buildID=2702646 and installed it manually.

Comment 6 Tad 2025-04-28 10:27:37 UTC
This also breaks firewalld when using the iptables backend:

Apr 28 06:21:46 localhost firewalld[1493]: WARNING: iptables-restore and iptables are missing, disabling IPv4 firewall.
Apr 28 06:21:46 localhost firewalld[1493]: WARNING: ip6tables-restore and ip6tables are missing, disabling IPv6 firewall.
Apr 28 06:21:46 localhost firewalld[1493]: WARNING: ebtables-restore and ebtables are missing, disabling bridge firewall.
Apr 28 06:21:48 localhost firewalld[1493]: ERROR: Failed to load user configuration. Falling back to full stock configuration.
Apr 28 06:21:48 localhost firewalld[1493]: ERROR: UNKNOWN_ERROR: No IPv4 and IPv6 firewall.

Comment 7 Alessandro 2025-04-28 11:34:45 UTC
Can confirm. latest 1.8.11-6 once again introduces many networking bugs.

Comment 8 Eric Garver 2025-04-28 12:57:19 UTC
Can users here share some logs? Mainly I want to see _where_ docker/firewalld are looking for iptables. There was a recent bin/sbin unification in f42 and I suspect this ticket is fallout from that.

For firewalld, you can enable debug /etc/sysconfig/firewalld. Use --debug=9. Then share the log after restarting firewalld.

Comment 9 David Peebles 2025-04-28 13:26:02 UTC
The latest update to 8.11-6 also breaks nordvpn.  After the upgrade, all attempts to create a nordvpn connection fail.

Comment 10 Tad 2025-04-28 13:57:02 UTC
firewalld is looking in eg. /usr/sbin/iptables-restore

but even though rpm -ql iptables-nft shows it should contain eg. /usr/bin/iptables-restore, neither file actually exists

1.8.11-5 does contain eg. /usr/sbin/iptables-restore and actually exists on the filesystem but it is a symlink to eg. /etc/alternatives/iptables-restore

Comment 11 Dave Koberstein 2025-04-28 17:12:35 UTC
It looks like the alternatives symlinks for /usr/sbin/iptables and /usr/sbin/ip6tables were removed. I was able to get docker running again by running altnernatives --config iptables and reselecting the existing choice. That recreated the symlinks. A restart of docker didn't work instantly - I had to reboot.

There's a reference to a change being made in the last iptables-legacy package that seems to affect this:
https://fedora.pkgs.org/42/fedora-updates-x86_64/iptables-legacy-1.8.11-6.fc42.x86_64.rpm.html

2025-04-25 - Zbigniew Jedrzejewski-Szmek  <zbyszek.pl> - 1.8.11-6
- Keep symlinks managed by alternatives under /usr/bin

Upon recreating the symlinks, the are in /usr/sbin, not /usr/bin. Do we really want them in /usr/bin?

Comment 12 Oleh Vdovenko 2025-04-29 09:35:29 UTC
Running `alternatives --config iptables` fixed the issue for me as well.

Comment 13 Phil Sutter 2025-04-29 10:07:31 UTC
Hmm. Updating iptables on a fresh install of F42 does no harm apart from
'alternatives --display iptables' showing duplicate entries. When
system-upgrading from F41 to F42 though (which performs the sbin-bin-merge for
me), the symlinks to /etc/alternatives disappear - there is no 'iptables' nor
'iptables-restore' or 'iptables-save' anymore.

Comment 14 Fedora Update System 2025-04-29 11:35:11 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b630544731

Comment 15 Fedora Update System 2025-04-29 11:35:17 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b630544731

Comment 16 Fedora Update System 2025-04-29 11:35:24 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b630544731

Comment 17 Fedora Update System 2025-04-29 11:35:31 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b630544731

Comment 18 Fedora Update System 2025-04-29 11:35:38 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b630544731

Comment 19 Fedora Update System 2025-04-29 11:35:56 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b630544731

Comment 20 Pavel Studeník 2025-04-29 13:29:35 UTC
Hi, today I upgraded system Fedora 42 and docker service has started to fail.

>> systemctl restart docker
Job for docker.service failed because the control process exited with error code.
>> journalctl -f
dub 29 15:26:05 fedora dockerd[9835]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: failed to create NAT chain DOCKER: dub 29 15:26:05 fedora systemd[1]: COMMAND_FAILED: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable
Failed to start docker.service - Docker Application Container Engine.

iptables-libs-1.8.11-6.fc42.x86_64
iptables-nft-1.8.11-6.fc42.x86_6

Comment 21 Fedora Update System 2025-04-29 20:40:22 UTC
FEDORA-2025-b630544731 (iptables-1.8.11-7.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 22 dangnho99 2025-04-30 03:41:22 UTC
I've just updated to latest (iptables-1.8.11-7.fc42) and still cannot start docker

Comment 23 Dave Koberstein 2025-04-30 03:47:38 UTC
The new build should not break an upgrade but if your upgrade was broken, I don't think it will fix it. 

Try running the alternatives command mentioned above. I think that will fix it and hopefully future upgrades won't break it based on the recent fixes.

Comment 24 dangnho99 2025-04-30 05:33:37 UTC
(In reply to dangnho99 from comment #22)
> I've just updated to latest (iptables-1.8.11-7.fc42) and still cannot start
> docker

Strange. It's ok now. After let the laptop go to suspend to go having lunch, it works.

Comment 25 Phil Sutter 2025-04-30 07:59:53 UTC
(In reply to Dave Koberstein from comment #23)
> The new build should not break an upgrade but if your upgrade was broken, I
> don't think it will fix it. 
> 
> Try running the alternatives command mentioned above. I think that will fix
> it and hopefully future upgrades won't break it based on the recent fixes.

Yes, I expected the same. The reports from testers saying it fixed their setup
hence surprised me, but who knows, maybe they rolled back before testing.

(In reply to dangnho99 from comment #22)
> I've just updated to latest (iptables-1.8.11-7.fc42) and still cannot start
> docker

As suggested, you may have to reapply your alternatives config. Either way, no
need to cancel my needinfo request because of that! ;)

Comment 26 Zbigniew Jędrzejewski-Szmek 2025-05-13 09:00:05 UTC
Yeah, the existing fixes were incomplete. I submitted a new round of pull requests now:
https://src.fedoraproject.org/rpms/iptables/pull-request/14
https://src.fedoraproject.org/rpms/iptables/pull-request/15

Comment 27 Francesco Bucciantini 2025-06-01 19:21:14 UTC
It looks like this is also affecting ufw.
ufw 0.35-33 (latest version) doesn't seem to be able to start correctly with iptables-nft 1.8.11-8.fc42, however downgrading to version 1.8.11-4.fc42 solves the issue.

Comment 28 fedora 2025-06-21 15:16:29 UTC
Fix is still uncomplete in iptables-nft-1.8.11-8.fc42.x86_64

Updated today from F40 to F42.
Same observation: docker daemon is not starting. 

Logfile:
=======
Jun 21 14:54:49 nas dockerd[2436]: time="2025-06-21T14:54:49.186879702+02:00" level=warning msg="failed to find iptables" error="exec: \"iptables\": executable file not found in $PATH"
Jun 21 14:54:49 nas dockerd[2436]: time="2025-06-21T14:54:49.241449401+02:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby

Packages:
=========
$ rpm -qa | grep iptables-nft
iptables-nft-1.8.11-8.fc42.x86_64

$ rpm -qa | grep docker-ce
docker-ce-cli-28.2.2-1.fc42.x86_64
docker-ce-rootless-extras-28.2.2-1.fc42.x86_64
docker-ce-28.2.2-1.fc42.x86_64

Workaround:
===========
Running as workaround command from above and reboot fixes the issue here, too
$ sudo alternatives --config iptables

Comment 29 Phil Sutter 2025-06-24 11:10:27 UTC
Reopening as per comment 28. Zbigniew, this is yours now.

Comment 30 Wainer dos Santos Moschetta 2025-07-11 20:14:50 UTC
(In reply to fedora from comment #28)
> Fix is still uncomplete in iptables-nft-1.8.11-8.fc42.x86_64
> 
> Updated today from F40 to F42.
> Same observation: docker daemon is not starting. 
> 
> Logfile:
> =======
> Jun 21 14:54:49 nas dockerd[2436]:
> time="2025-06-21T14:54:49.186879702+02:00" level=warning msg="failed to find
> iptables" error="exec: \"iptables\": executable file not found in $PATH"
> Jun 21 14:54:49 nas dockerd[2436]:
> time="2025-06-21T14:54:49.241449401+02:00" level=info msg="stopping event
> stream following graceful shutdown" error="<nil>" module=libcontainerd
> namespace=moby
> 
> Packages:
> =========
> $ rpm -qa | grep iptables-nft
> iptables-nft-1.8.11-8.fc42.x86_64
> 
> $ rpm -qa | grep docker-ce
> docker-ce-cli-28.2.2-1.fc42.x86_64
> docker-ce-rootless-extras-28.2.2-1.fc42.x86_64
> docker-ce-28.2.2-1.fc42.x86_64
> 
> Workaround:
> ===========
> Running as workaround command from above and reboot fixes the issue here, too
> $ sudo alternatives --config iptables

Same here. Upgraded from F40 to F42 and had docker-ce installed.
Then I switched to docker packaged by Fedora, rebooted, but got the same error.
I'd like to avoid the workaround of `sudo alternatives --config iptables` because I don't know if it can bring bad consequences on other parts of my system and/or on future upgrades of version.

Comment 31 Fedora Release Engineering 2026-05-06 12:46:54 UTC
This message is a reminder that Fedora Linux 42 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 42 on 2026-05-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '42'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 42 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.


Note You need to log in before you can comment on or make changes to this bug.