Bug 743281 - Disk encryption with national keymap doesn't work
Disk encryption with national keymap doesn't work
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
19
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Vratislav Podzimek
Fedora Extras Quality Assurance
RejectedBlocker, https://fedoraprojec...
: CommonBugs
Depends On: 837292
Blocks:
  Show dependency treegraph
 
Reported: 2011-10-04 09:31 EDT by Kamil Páral
Modified: 2015-01-13 09:18 EST (History)
17 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-01-13 09:18:16 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
disk encryption in anaconda 18.19 (25.56 KB, image/png)
2012-10-24 15:20 EDT, Kamil Páral
no flags Details

  None (edit)
Description Kamil Páral 2011-10-04 09:31:55 EDT
Description of problem:
I have followed this test case:
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_(encrypted)_install

I have done a default Live install. I have chosen "Czech (qwerty)" keyboard layout and selected "Encrypt system" check box. I have chosen "ěščřžýáíé" as the password, which is the same sequence as if you type "234567890" on an English keyboard layout.

After installation the system rebooted and I am not able to unlock the system during boot. It always complains about invalid password. When I see my grub options, it contains "KEYTABLE=cz-lat2" option, which should be correct.

I also tried anaconda rescue mode, I selected cz-lat2 keyboard layout which should be another name for "Czech (qwerty)", but I am also unable to unlock my partition. I can't even unlock it from anaconda rescue system shell manually.

Version-Release number of selected component (if applicable):
anaconda-16.20
Fedora 16 Beta Final i386 Live

How reproducible:
always

Steps to Reproduce:
1. install from Live (other methods untested)
2. select "Czech (qwerty)" layout
3. use "ěščřžýáíé" as the password (equivalent of "234567890" on the english layout)
4. boot
  
Additional info:
Due to very unfortunate decision of the Anaconda team not to allow user to display his own password, I am not able to verify that I'm actually entering what I want to be entering, in any of these steps:
1. disk encryption during installation
2. system boot
3. disk unlocking during rescue mode

Therefore I can't say whether I:
1. used the intended password for disk encryption in the first case
2. am unlocking the disk with the intended password or using different characters
Comment 1 Kamil Páral 2011-10-04 09:35:18 EDT
Proposing as F16 Blocker, because it fails multiple release criteria (Alpha#11, Alpha#14, Beta#12 and probably some others).
Comment 2 Kamil Páral 2011-10-04 10:08:36 EDT
I reproduced also for i386 DVD minimal install. The same issue.
Comment 3 Kamil Páral 2011-10-04 10:57:04 EDT
I want to highlight that unlocking doesn't work even in anaconda rescue mode, so I am not sure whether this is anaconda bug, dracut bug, or other software bug.
Comment 4 Adam Williamson 2011-10-04 16:51:25 EDT
this falls under the "There may be times where a requirement is unmet only in a particular configuration, such as with some keyboard layouts but not others, or if a particular character is used in a username, password or passphrase. In such cases, the release team should use their judgement and refer to precedent to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by the issue, the severity of the case when the issue is encountered, and the ease or otherwise with which the issue can be avoided by both informed and uninformed users. " wiggle clause in the criteria, but I'm +1 blocker due to the severity of the impact.
Comment 5 Kamil Páral 2011-10-05 03:10:03 EDT
Another note: All tests were performed on bare metal. This is not VM related.
Comment 6 Kamil Páral 2011-10-05 05:28:23 EDT
I tested several other layouts (with completely the same character sequence). The results are:
"Czech (qwerty)" doesn't work
"Czech" works
"Slovak (qwerty)" doesn't work
"French" works

I couldn't test more because my CDROM seems to stop working. Maybe later.
Comment 7 Kamil Páral 2011-10-05 07:10:19 EDT
"Belgian (be-latin1)" works

I didn't find more layouts that would differ from US layout on 234567890 characters and be available on Live CD. It is possible that some more layouts doesn't work that differ on different keys.
Comment 8 Kamil Páral 2011-10-05 07:26:04 EDT
"Turkish" doesn't work

They have the same characters on 234567890 as US, but they differ on A-Z characters, so I used a password from that area.
Comment 9 Harald Hoyer 2011-10-05 10:14:12 EDT
seems like a discrepancy of X11 and KEYBOARD... 

(In reply to comment #3)
> I want to highlight that unlocking doesn't work even in anaconda rescue mode,

-> not a dracut bug
Comment 10 Harald Hoyer 2011-10-05 10:14:44 EDT
moving back to anaconda
Comment 11 Adam Williamson 2011-10-07 13:33:14 EDT
Discussed at 2011-10-07 blocker review meeting. Accepted as a blocker per criterion "Following on from the previous criterion, after firstboot is completed and on subsequent boots, a system installed according to any of the above criteria (or the appropriate Beta or Final criteria, when applying this criterion to those releases) must boot to a working graphical environment without unintended user intervention. This includes correctly accessing any encrypted partitions when the correct passphrase is entered". This is a 'grey area' bug as it relates to specific languages, but we decided it was severe enough to merit blocker status.

Thoughts from the meeting:

<clumens> i don't see how it's an anaconda thing.
 unless the keyboard variant just isn't getting set at all, which should be easy to check elsewhere too
<dgilmore> that should be testable by setting the keyboard to us and hitting tah same keys
<dgilmore> if it unlocks keyboard is not set right when the encryption is setup

I've CCed who-t and drago01 in the hopes that they'll have some ideas here, being the s-s-k experts. What's the difference between the layouts that work and those that don't?
Comment 12 Kamil Páral 2011-10-10 02:34:41 EDT
According to my testing this is also reproducible inside KVM.

(In reply to comment #11)
> <dgilmore> that should be testable by setting the keyboard to us and hitting
> tah same keys
> <dgilmore> if it unlocks keyboard is not set right when the encryption is setup

When I use "Czech (qwerty)" to lock the disk, it can't be unlocked with neither "cz-lat2" nor "us" keymap.
Comment 13 Adel Gadllah 2011-10-10 04:52:32 EDT
(In reply to comment #11)

> I've CCed who-t and drago01 in the hopes that they'll have some ideas here,
> being the s-s-k experts. What's the difference between the layouts that work
> and those that don't?

Well this is in early boot so I'd guess it is either a systemd or a plymouth bug.
Comment 14 Adam Williamson 2011-10-14 14:20:54 EDT
Could this be caused by https://bugzilla.redhat.com/show_bug.cgi?id=744641 ?
Comment 15 Peter Hutterer 2011-10-17 20:30:09 EDT
(In reply to comment #14)
> Could this be caused by https://bugzilla.redhat.com/show_bug.cgi?id=744641 ?

doubt it. the s-s-k bug affects X only which isn't running this early in the boot stage.
Comment 16 Adam Williamson 2011-10-17 23:38:16 EDT
but X is running when you _set_ the passwords, during install or firstboot...
Comment 17 Peter Hutterer 2011-10-18 02:03:00 EDT
hmm, good point, didn't think of that. and the amount of typing needed during install is small enough that a misbehaving keymap may not be detected.
Comment 18 Adam Williamson 2011-10-18 02:50:27 EDT
Has anyone tested typing a 'foreign language' string in a dialog box where you can actually *see* what's being typed during installation, to see if the 'correct' map is used during install or not? Try the "Hostname" dialog.
Comment 19 Adam Williamson 2011-10-18 02:53:23 EDT
okay, I tested it: if I pick Czech (Qwerty) and type 234567890 into the hostname dialog I get ěščřžýáíé . So it looks like the right map is used during installation.
Comment 20 Kamil Páral 2011-10-18 07:38:30 EDT
For all national keymaps I tested I tried them out in the Hostname text field (that's the only field where you can test your keys). They keymap appears to set correctly at that stage. Of course that doesn't necessarily mean that the correct characters are input into the password dialog, but it is likely.
Comment 21 Ales Kozumplik 2011-10-19 03:31:11 EDT
There are two problems:

1) (something used by) dracut does not set the proper keyboard for those layouts where Kamil's tests failed (e.g. cz-lat2 fails, but not cz-us-qwertz is ok). This is easy to verify on any luks installation, use 'rdshell rdinitdebug KEYTABLE=cz-lat2' and once dracut drops you into shell you'll see that you are unable to type the accented characters from the top row (numbers are output).

2) the Anaconda rescue mode keyboard is even more broken and even cz-us-qwertz outputs weird, non-czech, characters.

I'll keep this bug opened and see what I can do about the rescue issue.
Comment 22 Harald Hoyer 2011-10-19 05:28:25 EDT
(In reply to comment #21)
> There are two problems:
> 
> 1) (something used by) dracut does not set the proper keyboard for those
> layouts 

dracut uses normal "loadkeys"

Switch to the console and try:

# loadkeys -q -u cz-lat2
and
# loadkeys -q -u cz-us-qwertz

and see, if this works ... otherwise it might be a bug in 

$ rpm -qf /lib/kbd/keymaps/i386/qwerty/cz-lat2.map.gz
kbd-misc-1.15.2-3.fc16.noarch
Comment 23 Harald Hoyer 2011-10-19 05:29:48 EDT
or just don't use "cz-lat2"
Comment 24 Ales Kozumplik 2011-10-19 06:56:56 EDT
(In reply to comment #22)
> (In reply to comment #21)
> > There are two problems:
> > 
> > 1) (something used by) dracut does not set the proper keyboard for those
> > layouts 
> 
> dracut uses normal "loadkeys"
> 
> Switch to the console and try:
> 
> # loadkeys -q -u cz-lat2
> and
> # loadkeys -q -u cz-us-qwertz
> 
> and see, if this works ... otherwise it might be a bug in 
> 
> $ rpm -qf /lib/kbd/keymaps/i386/qwerty/cz-lat2.map.gz
> kbd-misc-1.15.2-3.fc16.noarch

Ah, Kamil and me think this is because cz-lat2 (and the other encodings) need to be switched with a special sequence to output the accented characters. But then so should graphical Anaconda.
Comment 25 Harald Hoyer 2011-10-19 07:02:57 EDT
And I think, they are not UTF-8 but:

$ zgrep charset /lib/kbd/keymaps/i386/qwerty/cz-lat2-prog.map.gz
charset "iso-8859-2"
Comment 26 Kamil Páral 2011-10-19 07:39:25 EDT
I have found out interesting results.

When I encrypt my disk using "cz-lat2" layout using "ěščřžýáíé" (equals 234567890) password, I can't decrypt it. We know that. But if I change KEYTABLE in grub to "cz-us-qwertz", I can decrypt it! Therefore the disk was encrypted with correct password, just the decrypting process uses invalid characters.

Now, I think Ales found the issue. When I install an unencrypted minimal system and boot with KEYTABLE="cz-us-qwertz" and log in as root in VT, the czech keymap is enabled by default and I can use Pause key to switch to english keymap.

But if I boot with KEYTABLE="cz-lat2" and log in as root in VT, the *english* layout is enabled by default and I have to use Pause key to switch to the czech layout.

Do you see that? The problem is that "cz-lat2" layout is loaded but not set as default.

Now, back to the encrypted system. I can't decrypt it with KEYTABLE=cz-lat2. But if I hit Pause before entering the password, the decryption works! Mystery solved, (at least for the czech layout) the problem lies in switching the layouts and setting the default one.
Comment 27 Ales Kozumplik 2011-10-19 08:26:52 EDT
(In reply to comment #26)
> Now, back to the encrypted system. I can't decrypt it with KEYTABLE=cz-lat2.
> But if I hit Pause before entering the password, the decryption works! Mystery
> solved, (at least for the czech layout) the problem lies in switching the
> layouts and setting the default one.

Right. This is only confusing because when 'cz-lat2' is selected, text console is switched into 'cz-lat2', that is English layout by default that can be switched to Czech by pressing Pause. We use system-config-keyboard to map the console keymap to an X keymap but in the case of cz-lat2 it maps it to one that is in Czech layout by default. Moving to sc-keyboard.
Comment 28 Adam Williamson 2011-10-21 15:13:25 EDT
Lubomir isn't _terribly_ active at present, it seems, and we have a tight timeframe for blockers now, we really need them fixed by Monday.

So, if anyone following this bug is able to provide an s-c-k patch, please do. Please try to fix all potentially-affected layouts, we know at least Czech (qwerty), Slovak (qwerty) and Turkish are affected.

Thanks!
Comment 29 Peter Hutterer 2011-10-23 23:02:36 EDT
I had another look at it and I don't have a good solution. The crux of the matter is simple: the password is defined while the keyboard has an xkb layout, the password is entered while the keyboard has a console layout.

The only way this can work is if both layouts are identical. XKB currently does not have a layout that matches the console layout for the above use-cases (I only looked at cz and sk, it was easier to understand the differences here).

There are a few options to fix this but neither is something for the last day of fixes:
- restrict the layouts in system-config-keyboard to those that match XKB options. This likely means dropping quite a few. Since some characters are different on all keyboards, I expect this to be a blurry line of which ones to drop.
- add XKB layouts that match the console layouts. This would have to be done on a one-by-one case and tested by users of these layouts. Modifying layouts is more-or-less out of the question, there is only so much hatemail I'm willing to deal with.
- add console layouts that match the XKB layouts. Same issue as above, and rather pointless IMO. A better course would be for the console to share XKB keyboard descriptions but that's a long term project.
- Anaconda could carry the console keymaps and drop the user to that keymap for the password entry. Short of dropping the user to a tty and then back to X, this would also require a console-matching XKB layout

So neither option is easy and none are for immediate fixes.
Comment 30 Ales Kozumplik 2011-10-24 02:11:37 EDT
I wouldn't worry too much about this bug, tested on F15 where the behaviour is the same and I don't see how this could ever work previously. I suggest moving this to F17 and then decide about one of the strategies from comment 29. There is a lot of work happening on the new Anaconda UI, perhaps there could be a disclaimer added notifying the user while entering luks passphrase that the console and xkb layouts are not the same.
Comment 31 Kamil Páral 2011-10-24 05:16:35 EDT
I had no idea there are two different sets of keyboard layouts, one for console and one for X, and they are different. That's really broken.

I don't see a reasonable solution for this, except of merging those layouts, which is a long run. Anyone has better idea than to put it into CommonBugs and advise users to use plain US keyboard if they want to use disk encryption in order to have maximum compatibility?
Comment 32 Adam Williamson 2011-10-24 13:25:32 EDT
Or just advise on the layouts to avoid, at least.

If this is a structural bug which has been around for a while, I agree on dropping blocker status and moving to fix it in F17, and documenting for F16.

Can we get other votes?

Revising to -1 blocker.
Comment 33 Tim Flink 2011-10-24 13:27:22 EDT
(In reply to comment #32)
> Or just advise on the layouts to avoid, at least.
> 
> If this is a structural bug which has been around for a while, I agree on
> dropping blocker status and moving to fix it in F17, and documenting for F16.
> 
> Can we get other votes?
> 
> Revising to -1 blocker.

Agreed. Revising to -1 blocker
Comment 34 Sandro Mathys 2011-10-24 13:38:20 EDT
Seems the sensible way to go, -1 blocker. Or rather +1 F17Blocker ;)
Comment 35 Adam Williamson 2011-10-24 20:05:05 EDT
Okay, we have three not blocker votes, so revising blocker status.
Comment 36 Adam Williamson 2011-10-24 21:56:11 EDT
"I had no idea there are two different sets of keyboard layouts, one for console
and one for X, and they are different. That's really broken."

Oh, it's much, much worse than that.

I highly recommend not diving into the keyboard layout rabbit hole. You'll be a changed man when you emerge. It's impressive, in a weird way, just how much complexity people have managed to engineer into the process "press a button and cause a character to appear on the screen" over the last few decades. =)
Comment 37 Kamil Páral 2012-03-01 09:55:05 EST
To recapitulate:

X and console keyboard layouts do not match. Disk is encrypted in X (anaconda), but decrypted in console (plymouth). Often this can make the user fail to decrypt his disk, because some keys differ.

What we can possibly do:

1. The only reasonable solution is to merge console and X keyboard layouts (as noted in comment 29 "A better course would be for the console to share XKB
keyboard descriptions"), but that's a long term project. Is there some bug reported against that, or some upstream effort?

2. I had a long brainstorming with Martin Sivak from Anaconda and we couldn't find at least a partially working solution (other than recommending all users to use the US keymap if they want to encrypt their disk). Still, I'm changing component to anaconda and release to rawhide (f18), because it would be great if anaconda guys could come up with something how to warn the user in their new UI when there's a probability that he won't be able to unlock his encrypted disk after the installation is complete.

In my view, it can be a better approach to admit that our technology is not perfect and we can't guarantee this functionality apart from a few selected cases (keymaps), than to silently fail user without warning. But that's for anaconda devs' consideration.

3. Third idea I have is to discuss this issue with plymouth maintainer Ray Strode, whether there is some way to help users using "a wrong keymap" while unlocking their disks.
Comment 38 Chris Lumens 2012-10-23 10:59:00 EDT
Vratislav has done a ton of work around this area for newui.  Can someone please double check and see if this is being handled to your satisfaction now?
Comment 39 Adam Williamson 2012-10-23 19:26:26 EDT
To be specific, when you say 'now', I'm assuming you mean Beta TC6, yes?
Comment 40 Chris Lumens 2012-10-24 10:24:05 EDT
I mean the most recent release available.  This bug has not been commented on since before newui got merged in.
Comment 41 Kamil Páral 2012-10-24 15:20:15 EDT
Created attachment 632958 [details]
disk encryption in anaconda 18.19

Currently this is very hard to test due to bug 861123. Before I enter all required passwords (for some reason it asks me for more then one encrypted partition, even though the password should be the same), it enters rescue mode.

But looking at Anaconda 18.19:
Good things:
* It says which keymap I use when entering the password
* It discourages from changing keyboard layout afterwards
Bad things:
* I can't test the keymap when entering the password (I can test it before in a separate dialog, but that's not the same).
* I can't display the password, so I can't ensure I have typed in the intended password
* It doesn't warn me that the console layout in Plymouth is often different from xkb layout in Anaconda and doesn't suggest safe variant (English US layout).

If you read comment 29 and comment 37, you'll see there really isn't much anaconda can do easily, except for displaying a reasonable warning message (saying that 1) console and xkb layouts often don't match 2) using US keyboard is recommended when encrypting disk 3) keyboard layouts can afterwards be adjusted per-user in the desktop environment). That is not included in anaconda 18.19 (screenshot attached). Therefore I don't see any anaconda changes directly related to this bug.
Comment 42 Adam Williamson 2013-02-26 21:29:53 EST
It's worth noting that https://bugzilla.redhat.com/show_bug.cgi?id=837292 ought to solve this.
Comment 43 Fedora End Of Life 2013-04-03 09:38:23 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
Comment 44 Fedora End Of Life 2015-01-09 11:48:34 EST
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 45 Kamil Páral 2015-01-13 09:18:16 EST
This works well in Fedora 21.

Note You need to log in before you can comment on or make changes to this bug.