Description of problem: qemu-user-static creates /proc/sys/fs/binfmt_misc/qemu-* files when running `dnf install qemu-user-static`. But when running `dnf remove qemu-user-static`, the created files are not removed. As a result, even after uninstalling qemu-user-static, below command works using qemu-user-static. ``` $ podman pull arm64v8/fedora $ podman run --rm -t arm64v8/fedora uname -m aarch64 ``` To remove those files to back to the state before `dnf install qemu-user-static`, we have to run below command by root. ``` # find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \; ``` I think /proc/sys/fs/binfmt_misc/ is a place where files are created by other programs or manually by people. So, I want to see the files are removed by below command when running "dnf uninstall qemu-user-static". ``` # find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \; ``` Here is the part. https://src.fedoraproject.org/rpms/qemu/blob/master/f/qemu.spec#_1345 ``` %postun user-static /bin/systemctl --system try-restart systemd-binfmt.service &>/dev/null || : ``` Version-Release number of selected component (if applicable): $ rpm -q qemu-user-static qemu-user-static-3.1.0-9.fc30.x86_64 How reproducible: Steps to Reproduce: Good to open 2 terminals to run by root and user. 1. Initialize files at /proc/sys/fs/binfmt_misc . ``` # find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \; $ ls /proc/sys/fs/binfmt_misc register status ``` 2. Running by command, /proc/sys/fs/binfmt_misc/qemu-* files are added by "/bin/systemctl --system try-restart systemd-binfmt.service". "systemctl start systemd-binfmt" will add the /proc/sys/fs/binfmt_misc/qemu-* files based on /usr/lib/binfmt.d/qemu-*-static.conf files in qemu-user-static RPM package. ``` $ sudo dnf install qemu-user-static ``` 3. /proc/sys/fs/binfmt_misc/qemu-* files are created. ``` $ ls /proc/sys/fs/binfmt_misc | cat qemu-aarch64 qemu-aarch64_be ... qemu-xtensaeb register status ``` 4. Each files are added with "flag: F". F flag is for persistent. That keeps the qemu-user-staic on memory. I love the F option. Please do not change the setting. https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html ``` $ cat /proc/sys/fs/binfmt_misc/qemu-aarch64 enabled interpreter /usr/bin/qemu-aarch64-static flags: F offset 0 magic 7f454c460201010000000000000000000200b700 mask ffffffffffffff00fffffffffffffffffeffffff ``` 5. Run testing program. "arm64v8/fedora" is a container to run on aarch64. (My host is x86_64). ``` $ uname -m x86_64 $ podman pull arm64v8/fedora $ podman run --rm -t arm64v8/fedora uname -m aarch64 ``` 6. Uninstall qemu-user-staic ``` $ sudo dnf remove qemu-user-static ``` 7. /proc/sys/fs/binfmt_misc/qemu-* files are still remaining. This behavior is not intuitive. I also asking it to systemd. [1] ``` $ ls /proc/sys/fs/binfmt_misc | cat qemu-aarch64 qemu-aarch64_be ... qemu-xtensaeb register status ``` 8. As a result, below command still works. ``` $ podman run --rm -t arm64v8/fedora uname -m aarch64 ``` /usr/bin/qemu-aarch64-static is removed because the file is a part of qemu-user-static. But maybe by the "flags: F", it is still loaded on the memory. I assume that is why "podman run --rm -t arm64v8/fedora uname -m" still works. ``` $ ls /usr/bin/qemu-aarch64-static ls: cannot access '/usr/bin/qemu-aarch64-static': No such file or directory ``` ``` $ cat /proc/sys/fs/binfmt_misc/qemu-aarch64 enabled interpreter /usr/bin/qemu-aarch64-static flags: F offset 0 magic 7f454c460201010000000000000000000200b700 mask ffffffffffffff00fffffffffffffffffeffffff ``` 9. After manually remove the files, "podman run --rm -t arm64v8/fedora uname -m " does not work correctly. The state back to the before installing qemu-user-static. But I do not want to run "# find ..." to remove the files manually. ``` # find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \; $ ls /proc/sys/fs/binfmt_misc | cat register status $ podman run --rm -t arm64v8/fedora uname -m standard_init_linux.go:207: exec user process caused "exec format error" ``` Actual results: ``` $ sudo dnf remove qemu-user-static $ podman run --rm -t arm64v8/fedora uname -m aarch64 ``` Expected results: ``` $ sudo dnf remove qemu-user-static $ podman run --rm -t arm64v8/fedora uname -m standard_init_linux.go:207: exec user process caused "exec format error" ``` Additional info: * [1] I opened the ticket for systemd related to this issue. https://github.com/systemd/systemd/issues/13129 * [2] binfmt-misc: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
How about adding /proc/sys/fs/binfmt_misc/qemu-* files to "%files user-static" of qemu.spec? I think this is an ideal way to solve this issue, if we can. On the way, /proc/sys/fs/binfmt_misc/qemu-* files are created in %install section by a command. But the problem is that a command to create the files such as "/bin/systemctl --system try-restart systemd-binfmt.service" may have to be executed by root user in %install section. But the user to build SRPM on mock environment is not "root" but "mockbuild" user. ``` $ git diff diff --git a/qemu.spec b/qemu.spec index ebdcb1d..3af76a2 100644 --- a/qemu.spec +++ b/qemu.spec @@ -1165,6 +1165,7 @@ do done popd +/bin/systemctl --system try-restart systemd-binfmt.service %endif # user_static @@ -1650,6 +1651,7 @@ getent passwd qemu >/dev/null || \ %{_exec_prefix}/lib/binfmt.d/qemu-*-static.conf %{_bindir}/qemu-*-static %{_datadir}/systemtap/tapset/qemu-*-static.stp +/proc/sys/fs/binfmt_misc/qemu-* %endif ```
Thanks for the report. I can't think of a package that points to /proc/... in its %files section, and in this case I don't know if that will even work because it's not going to find the files in the buildroot during package build AFAIK. I think the only way to handle this would be to essentially uninstall the binfmt files from /proc using postun or something. But systemd-binfmt doesn't seem to do that for us, and trying to do it by hand could mean that we legitimately remove some custom config a user may have set up for themselves if something goes wrong. So while this is behavior is not optimal, the impact seems quite small and IMO doesn't justify making the packaging more complex. What do you think?
> I can't think of a package that points to /proc/... in its %files section, and in this case I don't know if that will even work because it's not going to find the files in the buildroot during package build AFAIK. I agree adding /proc/ ... files in %files are not realistic for now. Fedora and the RPM build system are not ready for that. > I think the only way to handle this would be to essentially uninstall the binfmt files from /proc using postun or something. But systemd-binfmt doesn't seem to do that for us, and trying to do it by hand could mean that we legitimately remove some custom config a user may have set up for themselves if something goes wrong. I agree with "I think the only way to handle this would be to essentially uninstall the binfmt files from /proc using postun or something", though ideally I like to see that systemd-binfmt remove teh files. > So while this is behavior is not optimal, the impact seems quite small and IMO doesn't justify making the packaging more complex. What do you think? For me, the impact is big. Because there are packages that expect the content of "/proc/sys/fs/binfmt_misc/qemu-$cpu" is the content installed by qemu-user-static RPM. ``` $ cat /proc/sys/fs/binfmt_misc/qemu-aarch64 enabled interpreter /usr/bin/qemu-aarch64-static flags: F offset 0 magic 7f454c460201010000000000000000000200b700 mask ffffffffffffff00fffffffffffffffffeffffff ``` Below issue for mock command happens when flags setting is "flags: " (empty flag), not "flags: F". https://github.com/rpm-software-management/mock/issues/277 It is big impact when we use qemu-user-static RPM with below programs. The programs use https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh script internally to create /proc/sys/fs/binfmt_misc/qemu-$cpu files with different content. * https://github.com/multiarch/qemu-user-static * https://github.com/dbhi/qus There is a patch to add a function to qemu-binfmt-conf.sh to remove /proc/sys/fs/binfmt_misc/qemu-$cpu files. But it's not merged yet. * https://patchew.org/QEMU/20190312195009.GA6@99bbefa4bcea/ > * Option '-r|--clear' is added, which allows to remove an already registered target interpreter or a list of them. The implementation is functional but partial. Please, see the corresponding commit. * https://patchew.org/search?q=project%3AQEMU+qemu-binfmt-conf.sh We might be able to use this "qemu-binfmt-conf.sh -r" option to remove the /proc/sys/fs/binfmt_misc/qemu-$cpu files.
This message is a reminder that Fedora 30 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '30'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 30 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 30 changed to end-of-life (EOL) status on 2020-05-26. Fedora 30 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.