Bug 2338712

[root@zappa tmp]# which iptables
/usr/sbin/iptables
[root@zappa tmp]# which ip6tables
/usr/sbin/ip6tables

I'm not certain what's going on yet. Are you running the latest rawhide up to date? Or is this a rawhide package on older fedora?

This Change looks relevant:

  https://fedoraproject.org/wiki//Changes/Unify_bin_and_sbin

The latest firewalld build shows iptables, et al. being detected at /usr/bin instead of /usr/sbin.

  https://kojipkgs.fedoraproject.org//packages/firewalld/2.3.0/3.fc42/data/logs/noarch/build.log

checking for iptables... /usr/bin/iptables
checking for iptables-restore... /usr/bin/iptables-restore
checking for ip6tables... /usr/bin/ip6tables
checking for ip6tables-restore... /usr/bin/ip6tables-restore
checking for ebtables... /usr/bin/ebtables
checking for ebtables-restore... /usr/bin/ebtables-restore
checking for ipset... /usr/bin/ipset

I'm not sure how firewalld can find /usr/bin/iptables and find it on rawhide if the iptables-legacy doesn't create symlinks in /usr/bin/iptables . 

[root@zappa sbin]# rpm -qf iptables
iptables-legacy-1.8.11-3.fc42.x86_64


[root@zappa bin]# rpm -qf iptables-legacy
iptables-legacy-1.8.11-3.fc42.x86_64

And I've been on rawhide for the past few years .

[root@zappa config]# yum provides /usr/bin/iptables
Updating and loading repositories:
 Fedora rawhide - x86_64 - Updates                                                                                                                                                                    100% |  10.0 MiB/s |  61.4 MiB |  00m06s
 Fedora - Rawhide - Developmental packages for the next Fedora release                                                                                                                                100% | 122.4 KiB/s | 309.9 KiB |  00m03s
 Jenkins                                                                                                                                                                                              100% |  89.7 KiB/s | 132.9 KiB |  00m01s
Repositories loaded.
No matches found. If searching for a file, try specifying the full path or using a wildcard prefix ("*/") at the beginning.

[root@zappa config]# rpm -qal iptables-legacy
/usr/bin/ip6tables-legacy
/usr/bin/ip6tables-legacy-restore
/usr/bin/ip6tables-legacy-save
/usr/bin/iptables-legacy
/usr/bin/iptables-legacy-restore
/usr/bin/iptables-legacy-save
/usr/bin/iptables-xml
/usr/bin/xtables-legacy-multi
/usr/lib/.build-id
/usr/lib/.build-id/74
/usr/lib/.build-id/74/306fffdac5f8895cbc58e8629f9ff2c350ff50
/usr/sbin/ip6tables
/usr/sbin/ip6tables-restore
/usr/sbin/ip6tables-save
/usr/sbin/iptables
/usr/sbin/iptables-restore
/usr/sbin/iptables-save
/usr/share/man/man1/iptables-xml.1.gz
/usr/share/man/man8/xtables-legacy.8.gz
/usr/share/xtables
/usr/share/xtables/iptables.xslt

That's interesting ... it's in both but I don't see it in both location on the system.

You can ignore #6 ...

This is what I get if I try to build firewalld:
~~~
checking for iptables... /usr/sbin/iptables
checking for iptables-restore... /usr/sbin/iptables-restore
checking for ip6tables... /usr/sbin/ip6tables
checking for ip6tables-restore... /usr/sbin/ip6tables-restore
checking for ebtables... /usr/sbin/ebtables
checking for ebtables-restore... /usr/sbin/ebtables-restore
~~~

I think this indicates that the firewalld build in koji detects iptables in /usr/bin where as your machine iptables is detected in /usr/sbin. I think this is related to the merge of /usr/bin and /usr/sbin as indicated it comment 2.

Perhaps your system wasn't properly migrated to merge /usr/bin and /usr/sbin.
https://fedoraproject.org/wiki//Changes/Unify_bin_and_sbin#Scope

Can you check /usr? Is sbin a symlink to bin ?

(In reply to Eric Garver from comment #9)
> I think this indicates that the firewalld build in koji detects iptables in
> /usr/bin where as your machine iptables is detected in /usr/sbin. I think
> this is related to the merge of /usr/bin and /usr/sbin as indicated it
> comment 2.
> 
> Perhaps your system wasn't properly migrated to merge /usr/bin and /usr/sbin.
> https://fedoraproject.org/wiki//Changes/Unify_bin_and_sbin#Scope
> 
> Can you check /usr? Is sbin a symlink to bin ?


That's interesting .  On one of my rawhide hosts, I see a lot of symlinks in /usr/sbin/* pointing to their equivalent in /usr/bin/*.

[dhill@knox sbin]$ ls -altr iptables*
lrwxrwxrwx. 1 root root 26 Sep 14  2021 iptables -> /etc/alternatives/iptables
lrwxrwxrwx. 1 root root 31 Sep 14  2021 iptables-save -> /etc/alternatives/iptables-save
lrwxrwxrwx. 1 root root 34 Sep 14  2021 iptables-restore -> /etc/alternatives/iptables-restore
lrwxrwxrwx. 1 root root 25 Jan 17 19:26 iptables-translate -> ../bin/iptables-translate
lrwxrwxrwx. 1 root root 33 Jan 17 19:26 iptables-restore-translate -> ../bin/iptables-restore-translate
lrwxrwxrwx. 1 root root 24 Jan 17 19:26 iptables-nft-save -> ../bin/iptables-nft-save
lrwxrwxrwx. 1 root root 27 Jan 17 19:26 iptables-nft-restore -> ../bin/iptables-nft-restore
lrwxrwxrwx. 1 root root 19 Jan 17 19:26 iptables-nft -> ../bin/iptables-nft
lrwxrwxrwx. 1 root root 21 Jan 17 19:26 iptables-apply -> ../bin/iptables-apply
lrwxrwxrwx. 1 root root 27 Jan 17 19:26 iptables-legacy-save -> ../bin/iptables-legacy-save
lrwxrwxrwx. 1 root root 30 Jan 17 19:26 iptables-legacy-restore -> ../bin/iptables-legacy-restore
lrwxrwxrwx. 1 root root 22 Jan 17 19:26 iptables-legacy -> ../bin/iptables-legacy


Oh and now it's all there now in my "broken" system:
[root@zappa sbin]# ls -tlr iptables*
lrwxrwxrwx. 1 root root 34 May 22  2021 iptables-restore -> /etc/alternatives/iptables-restore
lrwxrwxrwx. 1 root root 26 May 22  2021 iptables -> /etc/alternatives/iptables
lrwxrwxrwx. 1 root root 31 May 22  2021 iptables-save -> /etc/alternatives/iptables-save
lrwxrwxrwx. 1 root root 21 Jan 17 19:41 iptables-apply -> ../bin/iptables-apply
lrwxrwxrwx. 1 root root 27 Jan 17 19:41 iptables-legacy-save -> ../bin/iptables-legacy-save
lrwxrwxrwx. 1 root root 30 Jan 17 19:41 iptables-legacy-restore -> ../bin/iptables-legacy-restore
lrwxrwxrwx. 1 root root 22 Jan 17 19:41 iptables-legacy -> ../bin/iptables-legacy

So I guess I should stop opening bugs for rawhide ...

I have 2 systems that were not properly migrated:
~~~
Perhaps your system wasn't properly migrated to merge /usr/bin and /usr/sbin.
~~~

There's 2 other rawhide servers that didn't merge /usr/sbin and /usr/bin.

I don't exactly know what migrates everything located in /usr/sbin to /usr/bin but here's a script I've ran in 2 of the environments and it appears to have worked well:
~~~
cd /usr/sbin
for p in *; do if [ ! -L $p ]; then mv $p ../bin; fi; done
mv grub2-setpassword ../bin
mv mkdict ../bin
cd ../
mv sbin sbin.old
ln -s bin sbin
cd bin
yum reinstall iptables-legacy
~~~

Note: if you use this because you hit the same issue, I'm not responsible for any issues you may hit .  Take  backup first.

I would guess this is an issue with Alternatives. The post install scriplet [1] appears to only create the sbin symlink if one does not already exist. Since you have "alternatives" symlinks then the symlink from /usr/sbin/foo to /usr/bin/foo is not created (overwriting the alternative to /etc/alternatives/foo). I think that's what's affecting your migration.

As for the firewalld RPM, maybe when the build is done in Koji it builds against iptables-nft and thus finds "iptables" in /usr/bin. With the iptables-legacy package "iptables" is in /usr/sbin and "iptables-legacy" is in /usr/bin. So the firewalld RPM paths are wrong when iptables-legacy is being used.

I'm not sure what the solution is here. I'll ping authors of the bin/sbin unification.

[1]: https://src.fedoraproject.org/rpms/filesystem/blob/rawhide/f/filesystem.spec#_246

The sbin merge works differently on fresh installs versus upgrades. On fresh installs, /usr/sbin itself is a symlink to /usr/bin . On upgrades, /usr/sbin is preserved as a separate directory, but things in it should become symlinks pointing to /usr/bin (as I understood it from Zbigniew's explanation, anyway). Zbigniew should look at this, I think, and see if it's a problem with the upgrade case.

Just updated a sleeping rawhide and found this in the output:

/usr/sbin cannot be merged, found /usr/sbin/accton
All files under /usr/local/sbin are symlinks; linking to ./bin...

Yeah, the situation with iptables is a bit of a mess.

https://src.fedoraproject.org/rpms/firewalld/pull-request/13 should
resolve the issue.

>>> Running trigger-post-uninstall scriptlet: 
filesystem-0:3.18-36.fc42.x86_64
/usr/sbin cannot be merged, /usr/sbin/VBoxService points to /opt/VBoxGuestAdditions-7.1.4/sbin/VBoxService

Another one, perhaps the filesystem package is the culprit and should be fixed ?   How is this preventing a merger ?   By the way, this will definitely break things up because some of the files installed by packages are in /usr/sbin with a checksum and then those are moved to /usr/bin with a different checksum which is older or newer than the other directory.


One quick solution I've done on 5 hosts so far:


1) from sbin move all ../bin/ symlink to sbin.old.symlnk
2) from sbin move all files and symlinks to ../bin except the existing ones
3) for each of the remaining files, move them to sbin.old.exist
4) yum reinstall (rpm -qf $existing_file)

It is not a bug if the merge is not completed. At this point, many packages haven't been successfully built for the mass rebuild, and still have the old paths under /usr/sbin. Thus, even on systems with only distro packages, the merge will not be finished. But that is expected and fine.

Once we have dealt with the distro packages, we can consider the options for 3rd party packages. But that's still a ways out.

FEDORA-2025-98113d824c (firewalld-2.3.0-4.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-98113d824c

FEDORA-2025-98113d824c (firewalld-2.3.0-4.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.