Description of problem: I have followed this test case: https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_(encrypted)_install I have done a default Live install. I have chosen "Czech (qwerty)" keyboard layout and selected "Encrypt system" check box. I have chosen "ěščřžýáíé" as the password, which is the same sequence as if you type "234567890" on an English keyboard layout. After installation the system rebooted and I am not able to unlock the system during boot. It always complains about invalid password. When I see my grub options, it contains "KEYTABLE=cz-lat2" option, which should be correct. I also tried anaconda rescue mode, I selected cz-lat2 keyboard layout which should be another name for "Czech (qwerty)", but I am also unable to unlock my partition. I can't even unlock it from anaconda rescue system shell manually. Version-Release number of selected component (if applicable): anaconda-16.20 Fedora 16 Beta Final i386 Live How reproducible: always Steps to Reproduce: 1. install from Live (other methods untested) 2. select "Czech (qwerty)" layout 3. use "ěščřžýáíé" as the password (equivalent of "234567890" on the english layout) 4. boot Additional info: Due to very unfortunate decision of the Anaconda team not to allow user to display his own password, I am not able to verify that I'm actually entering what I want to be entering, in any of these steps: 1. disk encryption during installation 2. system boot 3. disk unlocking during rescue mode Therefore I can't say whether I: 1. used the intended password for disk encryption in the first case 2. am unlocking the disk with the intended password or using different characters
Proposing as F16 Blocker, because it fails multiple release criteria (Alpha#11, Alpha#14, Beta#12 and probably some others).
I reproduced also for i386 DVD minimal install. The same issue.
I want to highlight that unlocking doesn't work even in anaconda rescue mode, so I am not sure whether this is anaconda bug, dracut bug, or other software bug.
this falls under the "There may be times where a requirement is unmet only in a particular configuration, such as with some keyboard layouts but not others, or if a particular character is used in a username, password or passphrase. In such cases, the release team should use their judgement and refer to precedent to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by the issue, the severity of the case when the issue is encountered, and the ease or otherwise with which the issue can be avoided by both informed and uninformed users. " wiggle clause in the criteria, but I'm +1 blocker due to the severity of the impact.
Another note: All tests were performed on bare metal. This is not VM related.
I tested several other layouts (with completely the same character sequence). The results are: "Czech (qwerty)" doesn't work "Czech" works "Slovak (qwerty)" doesn't work "French" works I couldn't test more because my CDROM seems to stop working. Maybe later.
"Belgian (be-latin1)" works I didn't find more layouts that would differ from US layout on 234567890 characters and be available on Live CD. It is possible that some more layouts doesn't work that differ on different keys.
"Turkish" doesn't work They have the same characters on 234567890 as US, but they differ on A-Z characters, so I used a password from that area.
seems like a discrepancy of X11 and KEYBOARD... (In reply to comment #3) > I want to highlight that unlocking doesn't work even in anaconda rescue mode, -> not a dracut bug
moving back to anaconda
Discussed at 2011-10-07 blocker review meeting. Accepted as a blocker per criterion "Following on from the previous criterion, after firstboot is completed and on subsequent boots, a system installed according to any of the above criteria (or the appropriate Beta or Final criteria, when applying this criterion to those releases) must boot to a working graphical environment without unintended user intervention. This includes correctly accessing any encrypted partitions when the correct passphrase is entered". This is a 'grey area' bug as it relates to specific languages, but we decided it was severe enough to merit blocker status. Thoughts from the meeting: <clumens> i don't see how it's an anaconda thing. unless the keyboard variant just isn't getting set at all, which should be easy to check elsewhere too <dgilmore> that should be testable by setting the keyboard to us and hitting tah same keys <dgilmore> if it unlocks keyboard is not set right when the encryption is setup I've CCed who-t and drago01 in the hopes that they'll have some ideas here, being the s-s-k experts. What's the difference between the layouts that work and those that don't?
According to my testing this is also reproducible inside KVM. (In reply to comment #11) > <dgilmore> that should be testable by setting the keyboard to us and hitting > tah same keys > <dgilmore> if it unlocks keyboard is not set right when the encryption is setup When I use "Czech (qwerty)" to lock the disk, it can't be unlocked with neither "cz-lat2" nor "us" keymap.
(In reply to comment #11) > I've CCed who-t and drago01 in the hopes that they'll have some ideas here, > being the s-s-k experts. What's the difference between the layouts that work > and those that don't? Well this is in early boot so I'd guess it is either a systemd or a plymouth bug.
Could this be caused by https://bugzilla.redhat.com/show_bug.cgi?id=744641 ?
(In reply to comment #14) > Could this be caused by https://bugzilla.redhat.com/show_bug.cgi?id=744641 ? doubt it. the s-s-k bug affects X only which isn't running this early in the boot stage.
but X is running when you _set_ the passwords, during install or firstboot...
hmm, good point, didn't think of that. and the amount of typing needed during install is small enough that a misbehaving keymap may not be detected.
Has anyone tested typing a 'foreign language' string in a dialog box where you can actually *see* what's being typed during installation, to see if the 'correct' map is used during install or not? Try the "Hostname" dialog.
okay, I tested it: if I pick Czech (Qwerty) and type 234567890 into the hostname dialog I get ěščřžýáíé . So it looks like the right map is used during installation.
For all national keymaps I tested I tried them out in the Hostname text field (that's the only field where you can test your keys). They keymap appears to set correctly at that stage. Of course that doesn't necessarily mean that the correct characters are input into the password dialog, but it is likely.
There are two problems: 1) (something used by) dracut does not set the proper keyboard for those layouts where Kamil's tests failed (e.g. cz-lat2 fails, but not cz-us-qwertz is ok). This is easy to verify on any luks installation, use 'rdshell rdinitdebug KEYTABLE=cz-lat2' and once dracut drops you into shell you'll see that you are unable to type the accented characters from the top row (numbers are output). 2) the Anaconda rescue mode keyboard is even more broken and even cz-us-qwertz outputs weird, non-czech, characters. I'll keep this bug opened and see what I can do about the rescue issue.
(In reply to comment #21) > There are two problems: > > 1) (something used by) dracut does not set the proper keyboard for those > layouts dracut uses normal "loadkeys" Switch to the console and try: # loadkeys -q -u cz-lat2 and # loadkeys -q -u cz-us-qwertz and see, if this works ... otherwise it might be a bug in $ rpm -qf /lib/kbd/keymaps/i386/qwerty/cz-lat2.map.gz kbd-misc-1.15.2-3.fc16.noarch
or just don't use "cz-lat2"
(In reply to comment #22) > (In reply to comment #21) > > There are two problems: > > > > 1) (something used by) dracut does not set the proper keyboard for those > > layouts > > dracut uses normal "loadkeys" > > Switch to the console and try: > > # loadkeys -q -u cz-lat2 > and > # loadkeys -q -u cz-us-qwertz > > and see, if this works ... otherwise it might be a bug in > > $ rpm -qf /lib/kbd/keymaps/i386/qwerty/cz-lat2.map.gz > kbd-misc-1.15.2-3.fc16.noarch Ah, Kamil and me think this is because cz-lat2 (and the other encodings) need to be switched with a special sequence to output the accented characters. But then so should graphical Anaconda.
And I think, they are not UTF-8 but: $ zgrep charset /lib/kbd/keymaps/i386/qwerty/cz-lat2-prog.map.gz charset "iso-8859-2"
I have found out interesting results. When I encrypt my disk using "cz-lat2" layout using "ěščřžýáíé" (equals 234567890) password, I can't decrypt it. We know that. But if I change KEYTABLE in grub to "cz-us-qwertz", I can decrypt it! Therefore the disk was encrypted with correct password, just the decrypting process uses invalid characters. Now, I think Ales found the issue. When I install an unencrypted minimal system and boot with KEYTABLE="cz-us-qwertz" and log in as root in VT, the czech keymap is enabled by default and I can use Pause key to switch to english keymap. But if I boot with KEYTABLE="cz-lat2" and log in as root in VT, the *english* layout is enabled by default and I have to use Pause key to switch to the czech layout. Do you see that? The problem is that "cz-lat2" layout is loaded but not set as default. Now, back to the encrypted system. I can't decrypt it with KEYTABLE=cz-lat2. But if I hit Pause before entering the password, the decryption works! Mystery solved, (at least for the czech layout) the problem lies in switching the layouts and setting the default one.
(In reply to comment #26) > Now, back to the encrypted system. I can't decrypt it with KEYTABLE=cz-lat2. > But if I hit Pause before entering the password, the decryption works! Mystery > solved, (at least for the czech layout) the problem lies in switching the > layouts and setting the default one. Right. This is only confusing because when 'cz-lat2' is selected, text console is switched into 'cz-lat2', that is English layout by default that can be switched to Czech by pressing Pause. We use system-config-keyboard to map the console keymap to an X keymap but in the case of cz-lat2 it maps it to one that is in Czech layout by default. Moving to sc-keyboard.
Lubomir isn't _terribly_ active at present, it seems, and we have a tight timeframe for blockers now, we really need them fixed by Monday. So, if anyone following this bug is able to provide an s-c-k patch, please do. Please try to fix all potentially-affected layouts, we know at least Czech (qwerty), Slovak (qwerty) and Turkish are affected. Thanks!
I had another look at it and I don't have a good solution. The crux of the matter is simple: the password is defined while the keyboard has an xkb layout, the password is entered while the keyboard has a console layout. The only way this can work is if both layouts are identical. XKB currently does not have a layout that matches the console layout for the above use-cases (I only looked at cz and sk, it was easier to understand the differences here). There are a few options to fix this but neither is something for the last day of fixes: - restrict the layouts in system-config-keyboard to those that match XKB options. This likely means dropping quite a few. Since some characters are different on all keyboards, I expect this to be a blurry line of which ones to drop. - add XKB layouts that match the console layouts. This would have to be done on a one-by-one case and tested by users of these layouts. Modifying layouts is more-or-less out of the question, there is only so much hatemail I'm willing to deal with. - add console layouts that match the XKB layouts. Same issue as above, and rather pointless IMO. A better course would be for the console to share XKB keyboard descriptions but that's a long term project. - Anaconda could carry the console keymaps and drop the user to that keymap for the password entry. Short of dropping the user to a tty and then back to X, this would also require a console-matching XKB layout So neither option is easy and none are for immediate fixes.
I wouldn't worry too much about this bug, tested on F15 where the behaviour is the same and I don't see how this could ever work previously. I suggest moving this to F17 and then decide about one of the strategies from comment 29. There is a lot of work happening on the new Anaconda UI, perhaps there could be a disclaimer added notifying the user while entering luks passphrase that the console and xkb layouts are not the same.
I had no idea there are two different sets of keyboard layouts, one for console and one for X, and they are different. That's really broken. I don't see a reasonable solution for this, except of merging those layouts, which is a long run. Anyone has better idea than to put it into CommonBugs and advise users to use plain US keyboard if they want to use disk encryption in order to have maximum compatibility?
Or just advise on the layouts to avoid, at least. If this is a structural bug which has been around for a while, I agree on dropping blocker status and moving to fix it in F17, and documenting for F16. Can we get other votes? Revising to -1 blocker.
(In reply to comment #32) > Or just advise on the layouts to avoid, at least. > > If this is a structural bug which has been around for a while, I agree on > dropping blocker status and moving to fix it in F17, and documenting for F16. > > Can we get other votes? > > Revising to -1 blocker. Agreed. Revising to -1 blocker
Seems the sensible way to go, -1 blocker. Or rather +1 F17Blocker ;)
Okay, we have three not blocker votes, so revising blocker status.
"I had no idea there are two different sets of keyboard layouts, one for console and one for X, and they are different. That's really broken." Oh, it's much, much worse than that. I highly recommend not diving into the keyboard layout rabbit hole. You'll be a changed man when you emerge. It's impressive, in a weird way, just how much complexity people have managed to engineer into the process "press a button and cause a character to appear on the screen" over the last few decades. =)
To recapitulate: X and console keyboard layouts do not match. Disk is encrypted in X (anaconda), but decrypted in console (plymouth). Often this can make the user fail to decrypt his disk, because some keys differ. What we can possibly do: 1. The only reasonable solution is to merge console and X keyboard layouts (as noted in comment 29 "A better course would be for the console to share XKB keyboard descriptions"), but that's a long term project. Is there some bug reported against that, or some upstream effort? 2. I had a long brainstorming with Martin Sivak from Anaconda and we couldn't find at least a partially working solution (other than recommending all users to use the US keymap if they want to encrypt their disk). Still, I'm changing component to anaconda and release to rawhide (f18), because it would be great if anaconda guys could come up with something how to warn the user in their new UI when there's a probability that he won't be able to unlock his encrypted disk after the installation is complete. In my view, it can be a better approach to admit that our technology is not perfect and we can't guarantee this functionality apart from a few selected cases (keymaps), than to silently fail user without warning. But that's for anaconda devs' consideration. 3. Third idea I have is to discuss this issue with plymouth maintainer Ray Strode, whether there is some way to help users using "a wrong keymap" while unlocking their disks.
Vratislav has done a ton of work around this area for newui. Can someone please double check and see if this is being handled to your satisfaction now?
To be specific, when you say 'now', I'm assuming you mean Beta TC6, yes?
I mean the most recent release available. This bug has not been commented on since before newui got merged in.
Created attachment 632958 [details] disk encryption in anaconda 18.19 Currently this is very hard to test due to bug 861123. Before I enter all required passwords (for some reason it asks me for more then one encrypted partition, even though the password should be the same), it enters rescue mode. But looking at Anaconda 18.19: Good things: * It says which keymap I use when entering the password * It discourages from changing keyboard layout afterwards Bad things: * I can't test the keymap when entering the password (I can test it before in a separate dialog, but that's not the same). * I can't display the password, so I can't ensure I have typed in the intended password * It doesn't warn me that the console layout in Plymouth is often different from xkb layout in Anaconda and doesn't suggest safe variant (English US layout). If you read comment 29 and comment 37, you'll see there really isn't much anaconda can do easily, except for displaying a reasonable warning message (saying that 1) console and xkb layouts often don't match 2) using US keyboard is recommended when encrypting disk 3) keyboard layouts can afterwards be adjusted per-user in the desktop environment). That is not included in anaconda 18.19 (screenshot attached). Therefore I don't see any anaconda changes directly related to this bug.
It's worth noting that https://bugzilla.redhat.com/show_bug.cgi?id=837292 ought to solve this.
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle. Changing version to '19'. (As we did not run this process for some time, it could affect also pre-Fedora 19 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This works well in Fedora 21.