I noticed that kernel or glibc package updates started to break Alt-Fn key handling. Normally, while Xorg is running, Alt-Fn key combinations do *not* switch the virtual console, and they can be used as shortcuts in the desktop environment (e.g. Alt-F4 to close a window). To switch the console while Xorg is active on the current console, one would have to press Ctrl-Alt-Fn, while Alt-Fn combinations still work on text mode console. However, starting a few weeks ago, after a kernel or glibc package update, Alt-Fn combinations are handled by Xorg and at the same time they switch the virtual console. I narrowed down the problem to the `systemctl daemon-reexec` command being run from the RPM scriptlets. For example, the glibc package runs this command at the end of the postinstall scriptlet. Reproducible: Always Steps to Reproduce: From a desktop environment, run `systemctl daemon-reexec` as root in a terminal. Actual Results: While the active virtual console is the Xorg session, Alt-Fn key combinations are handled both by Xorg and by the kernel (to switch the active console). Expected Results: While the active virtual console is the Xorg session, Alt-Fn key combinations are handled by Xorg only. A Ctrl-Alt-Fn key combination is required to switch the virtual console away from the Xorg session. I noticed that there is some interaction with the `kbd_mode` utility: [root@thinkpad-p1 ~]# kbd_mode The keyboard is in some unknown mode [root@thinkpad-p1 ~]# systemctl daemon-reexec [root@thinkpad-p1 ~]# kbd_mode The keyboard is in Unicode (UTF-8) mode Also, at this point, running `kbd_mode -k` seems to fix the issue. However, running `kbd_mode` again (with no parameters, to display the current mode) seems to crash Xorg. This could be a different bug.
systemd initializes /dev/console, but only on first boot, not on reexec. Could you run `perf trace -e ioctl 2>&1 | grep -F KDSKBMODE` alongside your reproducer? That should list all processes that change the keyboard mode of any console.
As it turns out, it's actually systemd :-/ For some reason, perf doesn't recognize KDSKBMODE, but still shows the ioctl request number in hex: [root@thinkpad-p1 ~]# perf trace -e ioctl 2>&1 | grep '0x4b.*0x45' 8466.001 ( 0.002 ms): systemd/1 ioctl(fd: 3, cmd: (NONE, 0x4b, 0x45, 0), arg: 0x3) = 0 FWIW: [root@thinkpad-p1 ~]# grep -rn KDSKBMODE /usr/include/ /usr/include/linux/kd.h:87:#define KDSKBMODE 0x4B45 /* sets current keyboard mode */ (and I got similar results with kbd_mode, which I could match against the strace log on kbd_mode). Also: [root@thinkpad-p1 ~]# strace -e ioctl -o log5 -p 1 [root@thinkpad-p1 ~]# grep KB log5 ioctl(3, KDSKBMODE, K_UNICODE) = 0 Then I tried something really bold, like: - install the debuginfo packages for glibc and systemd - attach with gdb to pid 1 and set a conditional breakpoint in ioctl() - run `systemctl daemon-reexec` (the reproducer) Unfortunately, gdb catches an internal error and crashes when systemd re-executes itself, and that's before it tries to run ioctl. Apparently debugging systemd is trickier than a regular program. If you have a better idea, please let me know. Thanks!
The only thing I can think of is that the command line filtering we do misbehaves somehow, thus early_skip_setup_check() fails to detect that systemd is just reexecuting. I see systemd in F37 is lagging a bit: there's v251.14 there but v251.18 is already available. In particular, v215.14 is missing this commit: https://github.com/systemd/systemd-stable/commit/a6ae06c41e50e458e7203e7e8806e58ed0837670 , which could be relevant if my assumption is correct. @rrendec Could you show us the content of /proc/1/cmdline? And if you're up for some experimenting, could you rebuild systemd with the patch mentioned above and try again? (Note that you'd have to reexec once, fixup the keyboard mode (as the reexec command line would still have been created by the old systemd) and then reexec again to check if it works.) @zbyszek Maybe an update is in order? There's not that much time to EOL left...
Thanks for the update! Sure, the systemd command line is included below. [root@thinkpad-p1 ~]# xargs -0 < /proc/1/cmdline /usr/lib/systemd/systemd rhgb --switched-root --system --deserialize=31 As for the missing commit, I'll rebuild later today and post an update here. I need to reboot my system anyway to take a kernel update, so I'll boot straight with the new systemd.
I tried the missing commit. No dice :( [root@thinkpad-p1 ~]# kbd_mode The keyboard is in some unknown mode [root@thinkpad-p1 ~]# systemctl daemon-reexec [root@thinkpad-p1 ~]# kbd_mode The keyboard is in Unicode (UTF-8) mode [root@thinkpad-p1 ~]# I rebooted the system *after* updating systemd, and I double-checked that the patch had been applied by looking at /usr/src/debug/systemd-251.14-2.fc37.x86_64/src/core/main.c (from systemd-debugsource-251.14-2.fc37.x86_64). The filter_args() call comes after the code that injects the args, as expected.
It turns out I can reproduce this... It is a regression between systemd-251.10-588.fc37 and systemd-251.14-2.fc37 .
Bisected to https://github.com/systemd/systemd-stable/commit/a6ae06c41e50e458e7203e7e8806e58ed0837670 .
Further bisection shows this has been fixed by https://github.com/systemd/systemd/commit/4f44d2c4f76922a4f48dd4473e6abaca40d7e555 (https://github.com/systemd/systemd-stable/commit/8c741666fa49a8c87910e2da7832dcf1f414e286 in v251-stable). Hence, all we need is to update to v251.18.
*** Bug 2249991 has been marked as a duplicate of this bug. ***
This message is a reminder that Fedora Linux 37 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '37'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 37 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
FEDORA-2023-1582a08b60 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-1582a08b60
FEDORA-2023-1582a08b60 has been pushed to the Fedora 37 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-1582a08b60` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-1582a08b60 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2023-1582a08b60 has been pushed to the Fedora 37 stable repository. If problem still persists, please make note of it in this bug report.