Bug 1890085 - English keyboard layout is erroneously used during initramfs phase (e.g. decrypting encrypted storage devices) instead of native layout, on Silverblue / ostree installs
Summary: English keyboard layout is erroneously used during initramfs phase (e.g. decr...
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: rawhide
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Jiri Konecny
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: https://discussion.fedoraproject.org...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-21 11:03 UTC by Joachim Frieben
Modified: 2024-05-07 16:03 UTC (History)
31 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:
bcotton: fedora_prioritized_bug+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1405539 0 unspecified NEW Changing the default keyboard layout changes also disk decryption in plymouth, but only after next kernel update (initra... 2024-04-19 15:56:18 UTC
Red Hat Bugzilla 2235728 0 unspecified CLOSED Keyboard layout for LUKS decryption prompt is English (US) 2023-09-08 20:25:30 UTC

Internal Links: 1405539

Description Joachim Frieben 2020-10-21 11:03:26 UTC
Description of problem:
After installing a recent snapshot of Fedora 33 Silverblue, the keyboard layout chosen during the installation is not retained in the kernel boot options. As a consequence, the default English keyboard layout is applied when editing the command line of the boot loader or entering the key of a LUKS-encrypted disk during the boot procedure.

Version-Release number of selected component (if applicable):
33.25.4-1.fc33

How reproducible:
Always

Steps to Reproduce:
1. Install current Fedora 33 Silverblue.
2. Boot system.
3. Try to edit boot options or to enter the key of a LUKS-encrypted disk.

Actual results:
The actual keyboard layout is the default English one and not the one chosen when installing th system.

Expected results:
Apply the keyboard layout chosen during the installation.

Additional info:
- Once the system has booted up, the correct keyboard layout is applied to virtual consoles as well as to the GNOME desktop environment including GDM.
- Current CentOS 8 Stream which employs anaconda-33.16.3.26-1.el8 is not affected by this issue.

Comment 1 Adam Williamson 2020-10-21 21:47:12 UTC
Well. There's a lot of detail involved in keyboard layouts.

The description here has some odd ideas. How do you expect a keyboard layout specified in "the kernel boot options" to take effect "when editing the command line of the boot loader"? The boot loader comes before the kernel, not after. :)

The layout used when entering an encryption passphrase is decided by the initramfs environment, produced by dracut. It's demonstrably not always English:

https://openqa.fedoraproject.org/tests/703303#step/disk_guided_encrypted_postinstall/1

that's an openQA test of the most recent Fedora 33 install, doing an encrypted install in Russian. As you can see, when booting the installed system, the layout used for entering the passphrase is ru, not en.

Ditto Japanese: https://openqa.fedoraproject.org/tests/703300#step/disk_guided_encrypted_postinstall/1
and French: https://openqa.fedoraproject.org/tests/703325#step/disk_guided_encrypted_postinstall/1

there may be a Silverblue-specific issue here; we don't run the non-English language encrypted install tests on the Silverblue installer image (though perhaps we could). Can you test with a non-Silverblue installer image and see if you see the same behaviour? Thanks.

Comment 2 Joachim Frieben 2020-10-22 18:03:05 UTC
(In reply to Adam Williamson from comment #1)
To be more specific, I have now compared Workstation and Silverblue install images of the Fedora 33 development branch labeled 20201021.n.0 by setting up respective systems.
After keeping English as the system language but changing the default and only keyboard layout from English to German in the graphical installer in both cases and performing the installation, I have booted both systems in text mode after removing "rhgb quiet" from the boot options.
1. The Workstation system behaves correctly by unlocking the disk after entering the LUKS password exactly as entered in the graphical installer. When booting the system in graphical mode, then the LUKS unlock screen indicates the keyboard layout correctly as German (DE).
2. The Silverblue system, however, requires the LUKS password to be entered assuming an English layout in order to unlock the disk. Accordingly, the graphical unlock screen does not indicate the German keyboard layout. After completing the boot procedure, both systems apply the German keyboard layout to both of virtual consoles and graphical login screen.
I have to correct myself insofar as the kernel boot options of both systems indeed do not contain any kind of keyboard or language setting. If I remember correctly this used to be the case some time in the past between 1997 and 2020 but is no more the case nowadays.

Comment 3 Adam Williamson 2020-10-22 18:07:58 UTC
OK, so basically this issue is specific to Silverblue. Or, probably, to ostree-based installs.

I have a hunch as to what's going on, I'll see later today if I can reproduce this and if it's broken the way I think it is.

Comment 4 Adam Williamson 2020-10-22 23:12:26 UTC
OK, so I can confirm the bug here indeed: I did an install from the F33 RC2 (==Final) Silverblue installer image, in French, with encryption enabled, and the installer used fr layout when creating the passphrase, but the installed system used en layout when decrypting.

The installed system itself has a correct /etc/vconsole.conf . The initramfs environment - as inspected by booting with rd.break and looking around - does not. I'm not sure if this is the issue yet and if so why it doesn't have one.

Comment 5 Adam Williamson 2020-10-22 23:26:58 UTC
OK, so I ran an equivalent install from the Server DVD (French, encryption enabled) and in that install, the initramfs environment has a /etc/vconsole.conf . So that's the difference. The question, I guess, is why.

Comment 6 Adam Williamson 2020-10-22 23:32:00 UTC
I ran 'dracut -f -v --debug' on both installs, grepped the output for 'vconsole', and compared the results; they're identical. That's odd.

Comment 7 Adam Williamson 2020-10-23 21:20:17 UTC
F33 final is signed off, so there's no point to FE status any more.

Comment 8 Vladimír Slávik 2020-11-12 18:21:06 UTC
Unfortunately, this is intentional as far as Anaconda's handling, see https://github.com/rhinstaller/anaconda/blob/master/pyanaconda/modules/storage/bootloader/installation.py#L187

Reassigning - hope that's the right component?

Comment 9 Adam Williamson 2020-11-12 18:30:37 UTC
well, the ostree package contains specific ostree tooling and isn't really an "ostree-based distributions" virtual component, but it got assigned to Colin so it's probably fine. :P

Comment 10 Jonathan Lebon 2020-11-12 19:08:28 UTC
I think this is a dupe of https://github.com/fedora-silverblue/issue-tracker/issues/3, which is now addressable by https://github.com/coreos/rpm-ostree/pull/2170 though it's still in experimental mode. Having it work seamlessly though will require some work on the Anaconda side to have it use the new API.

Comment 11 Adam Williamson 2020-11-12 19:20:59 UTC
Yes, that does look like the same issue indeed.

Comment 12 Vladimír Slávik 2020-11-12 19:31:04 UTC
Jonathan, if you think the API will be good enough for using in f34, feel free to switch the bug back to anaconda.

Comment 13 Jens Petersen 2021-05-10 12:37:26 UTC
Is now a good time to move the bug or did anything happen for F34?

Comment 14 Timothée Ravier 2021-05-10 17:01:37 UTC
https://github.com/fedora-silverblue/issue-tracker/issues/3#issuecomment-822689910 > Let's redirect this one to Anaconda

Comment 15 Vladimír Slávik 2021-05-13 14:49:42 UTC
Assuming I want to fix this, I want to do... what? Make Anaconda run this in chroot?

rpm-ostree ex initramfs-etc /etc/vconsole.conf

Or this?

rpm-ostree initramfs --enable
(and then something else since that changes the default?)

Wizards, please advise.



---

Notes to self or anyone trying to implement this. There are two alternatives:

a) Modify RecreateInitrdsTask.run() and plug in a new method depending on payload type. Not sure where exactly would the new method go.

b) Create a new ostree-specific Task, add it to the list in generate_initramfs_with_tasks() and make its run() conditional on payload type.

Not sure which is better.

Comment 16 Jonathan Lebon 2021-05-13 16:44:13 UTC
(In reply to Vladimír Slávik from comment #15)
> Assuming I want to fix this, I want to do... what? Make Anaconda run this in
> chroot?
> 
> rpm-ostree ex initramfs-etc /etc/vconsole.conf

Yes exactly, but rpm-ostree is a client and daemon which talk over D-Bus, so getting it set up will get messy. :) (We have a "peer" mode though that path has kinda fallen into disuse.)

It might be easier to actually not run it in a chroot and just use the rpm-ostree binary and system D-Bus already in the Anaconda environment (obviously, this introduces a version mismatch, though I don't think it will be hard for us to ensure config compatibility for this feature as long as the skew isn't massive). You'd start the daemon using something like `rpm-ostree start-daemon --sysroot /sysroot` and then you *should* be able to just use `rpm-ostree ex initramfs-etc`.

The other thing is that the command is still marked as experimental, which isn't ideal before integrating it into Anaconda. I'll look at promoting it to `rpm-ostree initramfs-etc`, but it shouldn't block hacking on this meanwhile.

Comment 17 Vladimír Slávik 2021-05-14 14:50:43 UTC
I tried to manually run the commands mentioned above, in the installation environment, once Rawhide Silverblue installation is finished.

- rpm-ostree start-daemon --sysroot /mns/sysroot -> ran for a bit then exited, journal has only one message that it's idle and will exit
- rpm-ostree ex initramfs-etc --track /etc/vconsole.conf -> error about system not booted via libostree
- rpm-ostree initramfs --enable -> also error

The error is the same with no daemon running, so I guess the client doesn't see it.

Then I rebooted into the new system and tried to run these again.

- rpm-ostree ex initramfs-etc --track /etc/vconsole.conf (also vconsole.conf) -> worked
- rpm-ostree initramfs --enable -> worked

Comment 18 Vladimír Slávik 2021-05-18 12:19:19 UTC
Related bug about config file missing in initramfs - https://bugzilla.redhat.com/show_bug.cgi?id=1801539#c2 - fstrim not working through luks. Can be fixed the same way.

Comment 19 Jonathan Lebon 2021-06-28 18:53:14 UTC
(In reply to Vladimír Slávik from comment #17)
> I tried to manually run the commands mentioned above, in the installation
> environment, once Rawhide Silverblue installation is finished.
> 
> - rpm-ostree start-daemon --sysroot /mns/sysroot -> ran for a bit then
> exited, journal has only one message that it's idle and will exit

Ahh right OK. So instead, you can try doing `systemctl edit rpm-ostreed` (or add the equivalent drop-in) to override `ExecStart` to be `/usr/bin/rpm-ostree start-daemon --sysroot /mnt/sysroot`. That way, you shouldn't have to start the daemon manually at all. We can probably clean this up to make it an env var like RPM_OSTREE_SYSROOT instead.

We discussed this recently upstream: https://github.com/coreos/rpm-ostree/pull/2854#discussion_r640720977. Long-term, it's possible that we'll have rpm-ostree "client commands" just take a `--sysroot` arg directly and not involve a daemon architecture at all. That should simplify things like this a lot too.

Comment 20 Jonathan Lebon 2021-06-28 18:55:32 UTC
For prototyping and dirty hacking, you can also keep running `rpm-ostree start-daemon --sysroot /mnt/sysroot` manually, but using RPMOSTREE_DEBUG_DISABLE_DAEMON_IDLE_EXIT=1, which is a debug option that prevents the daemon from auto-exiting on idle.

Comment 21 Ben Cotton 2021-08-10 12:48:48 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 22 David Kaspar // Dee'Kej 2022-01-24 17:01:55 UTC
Hello!

Any update on this issue? I can confirm having this issue with Fedora SB 35. :)

Comment 23 Vladimír Slávik 2022-01-25 17:41:18 UTC
initramfs-etc just recently got proposed as a stable command, I believe. If this is released before F35, perhaps we can finally resolve the bug, see comment 15.

Comment 24 Vladimír Slávik 2022-06-24 13:40:18 UTC
Hello Jonathan, has there been any progress on making initramfs-etc possible as a client command?

Note to self: bug 1801539 can be solved the same way.

Comment 25 Jonathan Lebon 2022-06-28 14:08:59 UTC
Hi Vladimír,

The `initramfs-etc` command is now out of experimental. However, no progress has been made on the "pure client" side.
Reading back, I think the issue you were hitting trying to stand up the daemon was

> - rpm-ostree ex initramfs-etc --track /etc/vconsole.conf -> error about system not booted via libostree

We should be able to hack around this.

Can you make sure that /run/ostree-booted is present in the chroot?

Comment 27 Miguel Ángel 2022-08-28 17:28:04 UTC
/run/ostree-booted is not present in the chroot

Comment 28 Ben Cotton 2022-11-29 16:49:54 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 29 David Kaspar // Dee'Kej 2022-11-29 17:59:52 UTC
Changing the affected Fedora version to F37, where I can still experience this bug (with fresh F37 Silverblube install), so that this bug is not automatically closed.

Comment 30 jonas 2023-01-24 08:46:56 UTC
Confirmed in F37 Silverblue. 

Displaying the keyboard layout (similar to F37 Workstation) would surely help.

Comment 31 Andrei Manzhov 2023-03-04 21:06:45 UTC
Confirmed also in F39 Silverblue (tried Gnome x86_64-Rawhide-20230303 and Kinoite x86_64-Rawhide-20230303).

In F37 Workstation the custom keyboard selection was both applied during setup AND shown on boot.
in all versions of Silverblue it is neither applied nor shown.

considering this thread's history, seems like it's time to update the version number once again :(

Comment 32 Adam Williamson 2023-03-04 21:44:56 UTC
yup, let's kick it to 38 at least.

Comment 33 Ben Cotton 2023-04-19 14:54:04 UTC
In today's prioritized bugs meeting, we agreed to accept this as a prioritized bug. Using the wrong keyboard layout during the boot process is a bad user experience
https://meetbot.fedoraproject.org/fedora-meeting-1/2023-04-19/fedora_prioritized_bugs_and_issues.2023-04-19-14.00.log.html#l-44

Comment 34 Timothée Ravier 2023-11-10 11:06:09 UTC
Another (maybe easier) option is to add arguments to the kernel command line to set the keymap. For example from https://www.freedesktop.org/software/systemd/man/latest/vconsole.conf.html:

```
... vconsole.keymap=fr
```

Comment 35 Joachim Frieben 2023-11-12 20:35:55 UTC
(In reply to Timothée Ravier from comment #34)
After adding "vconsole.keymap=de" to the boot parameters, the screen for entering the disk password indeed exhibits a keyboard symbol supplemented by language specifier "de", thanks!

Comment 36 Jonathan Lebon 2023-11-14 16:23:12 UTC
(In reply to Timothée Ravier from comment #34)
> Another (maybe easier) option is to add arguments to the kernel command line
> to set the keymap. For example from
> https://www.freedesktop.org/software/systemd/man/latest/vconsole.conf.html:
> 
> ```
> ... vconsole.keymap=fr
> ```

Nice find! I didn't realize there was a karg for it.
So one thing Anaconda could much more easily do then is to inject the right karg at install time.

Comment 37 David Kaspar // Dee'Kej 2023-11-15 21:32:13 UTC
This bug is here for 3 years now, and it significantly degrades the user experience for Fedora Silverblue users who do not use English keyboard layout. Could we please finally fix it? Fix in F40 would be much welcome...

-- Dee'Kej --

Comment 38 Joachim Frieben 2023-11-16 06:09:10 UTC
(In reply to Timothée Ravier from comment #34)
If have checked adding "vconsole.keymap=de" to the boot options of system with a disk encrypted using a German keyboard layout and a suitable password, and this boot option definitely resoves the issue.

Comment 39 Kamil Páral 2023-11-16 09:36:49 UTC
I believe vconsole.keymap= used to be present on kernel cmdline in the past, but it was then removed, and perhaps the reason was that when changing the default keyboard layout (on a non-atomic Fedora), it still overrode the change and forced the original keymap during encrypted disk unlocking. It's definitely something to check, before requesting it to be added back. There's also bug 1405539, which is somewhat related to this area. In general, this area of selecting and changing default keyboard layout is undercooked at this moment.

Comment 40 Jens Petersen 2024-01-04 08:19:54 UTC
This also affect F39, I assume?

Comment 41 Jiri Konecny 2024-01-04 13:12:56 UTC
Hi, unfortunately it is. Installer development team don't have capacity to handle this bug :(.

Comment 42 Keywan 2024-04-08 14:48:36 UTC
I'm not sure, if I should file a new bug or if it is the same bug. I experienced this problem with Fedora 40 Workstation (not Silverblue). 

I changed the language in the installer to German, then installed with LUKS with password "testz". But I must type "testy" during boot to unlock the luks partition. 

Then I started gnome-disks and tried to change the passphrase. If I tried "testz" as old luks passphrase, I couldn't change the luks passphrase, but with "testy" (in the GUI, gnome-disk with German environment). 

I didn't experienced this problem with older Fedora Workstation. 

Please let me know, if I can supply further information. In the next days I will reinstall Fedora several times, so I'm happy to test this again.

Comment 43 Adam Williamson 2024-04-08 15:09:44 UTC
that would be a different bug, but it seems strange, because we do have automated testing for that and it is not currently failing. see e.g. https://openqa.fedoraproject.org/tests/2552274 - that test installs in French, and for the encryption passphrase, it presses the keys that would produce the string 'weakpassword' on a US keyboard, which should give a different string in French (at least A becomes Q and W becomes Z, don't remember if anything else changes). It then presses the same keys when booting the installed system to unlock it. If the layout were different between the installer and the unlock prompt, the test would fail.

Theoretically the test can falsely pass if the layout is wrong at *both* points, but that's not what you describe, and we have a guard against that in the test (we have the system hit the keys marked 'qwerty' on a US keyboard when typing the username, and we check that the username actually comes out as 'azerty'; if not, the test fails).

Do let us know if you hit this consistently, though. Please file a new bug in that case.

Comment 44 Adam Williamson 2024-04-08 15:11:58 UTC
oh, wait, I just saw that you said "Workstation live". The live images do not attempt to change the keyboard layout; especially under Wayland it is awkward to have the installer (which is just an app running on the desktop) fighting with the desktop for control of the keyboard layout. On lives if you want to use a different layout you are supposed to configure it via the desktop environment. The KEYBOARD page tells you this if you go to it, I think.

We are working on a new Workstation live installer flow which goes through a language/layout selection wizard on boot (before you have the opportunity to reach the desktop or the installer) but it's not ready for F40, it's currently in Rawhide aimed for F41.

Comment 45 Keywan 2024-04-08 15:42:09 UTC
Ok, thanks for the info. Sorry, for the noise. Looking forward for the new installer.

Comment 46 Aoife Moloney 2024-05-07 15:42:42 UTC
This message is a reminder that Fedora Linux 38 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 38 on 2024-05-21.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '38'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 38 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.


Note You need to log in before you can comment on or make changes to this bug.