Bug 881624

Fedup doesn't care about keyboard layout at all. And some related configuration files changed between F17 and F18. I think most people have non-US keyboard layout.

I'm proposing this as a blocker, because fedup is recommended upgrade method as per http://fedoraproject.org/wiki/Upgrading

I think that one part of the problem is keyboard layout in initrd (encrypted disk unlocking). I see some warnings from dracut that SYSFONT= and KEYTABLE= is not supported on kernel command line.

The second part is X11. After upgrade, I've got English keyboard layout:

$ localectl 
   System Locale: LANG=cs_CZ.utf8
       VC Keymap: cz-lat2
      X11 Layout: n/a

Fedup should run something like this (with my previous config):

# localectl set-keymap cz-lat2
# localectl set-x11-keymap cz pc105 qwerty

well, no it shouldn't, necessarily. packages are meant to do that stuff themselves, and that's what 882212 was about. fedup runs package scripts just like normal, so that fix should apply to fedup upgrades.

After today's upgrades my keyboard layout for hdd unlock returned to German  again, so the hdd unlock issue seems to be fixed.

So after discussing this at today's blocker meeting, I *think* this is the situation:

Most F17->F18 keymap migration stuff should be handled by the systemd %post scripts, which will get run by fedup, so should work fine in a fedup-based upgrade once 882212 is fixed.

*But*, there's one significant bit which I don't think anything takes care of during upgrade: the kernel cmdline. After an F17 install with a non en-US keymap, cmdline has something like this:

KEYTABLE=fr-latin9

(that's a French install). For F18, apparently, this instead needs to be something like:

vconsole.keymap=fr

See https://bugzilla.redhat.com/show_bug.cgi?id=875567 . But I don't think we have any kind of %post script or magic in fedup to try and migrate the cmdline. So I think the "This is especially problematic if you use hard disk encryption with a password using special characters. I was unable to access the system due to this without checking for the US layout on a second PC." part of this may be valid even with 882212 fixed. Let's keep it open for that situation. I will test it.

Discussed at 2012-12-21 blocker review meeting: http://meetbot.fedoraproject.org/fedora-bugzappers/2012-12-21/f18final-blocker-review-7.2012-12-21-18.33.log.txt . If the problem with the cmdline exists as I described, this will be accepted as a blocker, because it would have significant impact on non-English installs with encrypted partitions: the keymap used for entering the passphrase would change to US. This doesn't only affect special characters - some perfectly normal characters are in different places on different keyboards. Imagine you use DVORAK, for instance. Or you use a French keyboard and have an encryption passphrase of qzmmzqa...point being, even Roman alphabet letters are not in the same place on all keyboard layouts. So this violates criterion "For each one of the release-blocking package sets ('minimal', and the package sets for each one of the release-blocking desktops), it must be possible to successfully complete an upgrade from a fully updated installation of the previous stable Fedora release with that package set installed, using any officially recommended upgrade mechanisms. The upgraded system must meet all release criteria."

I will try and test this with a sufficiently new upgrade.img and systemd package in the upgrade package set, to ensure nothing actually manages the migration.

So I'm having trouble getting an upgrade to fire, but I did try this: on an F18 system, edit cmdline to include KEYTABLE=fr-latin9 rd.break , and boot.

You'll get to a dracut console and the keymap will be French. It looks like dracut prints a deprecation warning for KEYTABLE= rather than vconsole.keymap= , but it still handles it properly. So for F18 I think we're probably okay on that front: it may need revisiting in future releases, though.

Not 100% sure if I should close this one as I can't get an upgrade to actually run to verify that the other migrations happen properly, but I think they ought to.

(In reply to comment #4)
> Most F17->F18 keymap migration stuff should be handled by the systemd %post
> scripts, which will get run by fedup, so should work fine in a fedup-based
> upgrade once 882212 is fixed.

OK, so the problem with layout in X11 should be fixed in systemd package:
https://bugzilla.redhat.com/show_bug.cgi?id=889699

Sorry for mixing it up with Fedup.

Does anyone still see a concern represented by this bug, esp. anything that blocks release? I don't think there is anything, but want to be sure before closing.

Hi,

can't tell as the problem was fixed on the updated version already. If you want I can set up a VM with Fedora 17 and try the upgrade again.

Regards,

Tom.

Just testing the update again. Some thing I've noticed: Running fedup behind a proxy fails when trying to download the boot image. The repolists are downloaded, but when trying to download the boot image the proxy is ignored. I've exported it as environment variable, after that the upgrade starts.
I'll provide a status update for the bug as soon as the update has finished.

Just rebooted the system to continue the upgrade process- same problem. After grub starts the upgrade entry, I can see for a short moment something like "Language=de not supported". Then when trying to access the hdd the keyboard is using UK layout. From my point of view still a blocker as the standard user is stuck at this point.

So, upgrade has been done. Here the results:

- when rebooting after fedup-cli, the upgrade process uses US keyboard for hdd decryption

- after the upgrade, German keyboard is used for hdd encryption

- The login shell and gnome desktop uses US Keyboard


So the bug is not fully fixed yet, just one out of three issues.

Regards,

Tom.

Discussed at 2013-01-02 blocker review meeting: http://meetbot.fedoraproject.org/fedora-bugzappers/2013-01-02/f18final-blocker-review-8.2013-01-02-17.03.log.txt . We're still not 100% sure of exactly what's broken and working here - the case we're most worried about is passphrase entry for encrypted volumes during the upgrade process, so we will focus testing on that. https://bugzilla.redhat.com/show_bug.cgi?id=890677 may be complicating things there: it may be that upgrades for encrypted installs just aren't working at all. We are delaying the evaluation once more until we can nail down what's working and what isn't and what needs to be fixed prior to release.

*** Bug 890543 has been marked as a duplicate of this bug. ***

Doing a test of a graphical French install I can confirm Thomas' points 2 and 3: after upgrade, console layout is French but X and GNOME are using en-US. Further, GNOME is offering Bambara as a second layout, that weird freaking bug again.

I'm not sure this should be assigned to fedup, though, at least not any more. Shouldn't systemd %post be managing the config migration?

localectl reports:

System Locale: LANG=fr_FR.UTF-8
VC Keymap: fr-latin9
X11 Layout: n/a

gsettings get org.gnome.desktop.input-sources sources:

[('xkb', 'us'), ('xkb', 'ml')]

Sigh. I should re-open Jan's bug for that stuff. This bug should cover Thomas':

"when rebooting after fedup-cli, the upgrade process uses US keyboard for hdd decryption"

re-setting back to that.

I'm a bit lost here. The login shell (gdm) will use whatever layouts are specified in your X server configuration. How was the original reporter's system configured to use the 'de' layout?

Rui,

I installed a Fedora 17 system, during the setup process I chose German as Language and as Keyboard Layout.

After that, hdd-decrypt and GDM used German Keyboard Layot. When running Fedup, the results where as desribed in #12.

I am just reinstalling a VM with Fedora 17, if you need any configuration files or debug logs, I will be happy to provide.

I'd like to know the contents of /etc/X11/xorg.conf and any /etc/X11/xorg.conf.d/* files. Both right before the upgrade and right after it.

Rui: sorry, please follow up in https://bugzilla.redhat.com/show_bug.cgi?id=889699 for the post-upgrade X config stuff, that's the correct bug for that one.

This bug is for the keymap used during the second stage of upgrade, which is well before X gets started - it mostly affects cases where there's an encrypted partition which needs to be unlocked for the upgrade process to complete.

Sorry again for confusing the bug.

Hi!

I was affected by the same bug when i used fedup to uppgrade my f17 system, exactly like thomas.michel describes it. Only difference is that i had swedish layout before, but ended up with us layout just as him.

Discussed at 2012-01-03 go/no-go meeting: http://meetbot.fedoraproject.org/fedora-meeting-1/2013-01-03/f18_final_gono-go_meeting.2013-01-03-17.01.log.txt . Rejected as a blocker: it kind of sucks, but it should be documentable and workaroundable, and it may be fixable post-release (or outside of the freeze/spin cycle) with an update of F17's fedup.

Agreed, it seems to be a problem of fedup and should be fixed in there. I digged a little deeper into it and I think what happens is this:

Within the script /usr/lib/python2.7/site-packages/fedup/grubby.py the new grub boot entry is created. It just copies any command lines existing in the default entry to the new boot command. 
The part of the script is:

   def add_entry(self, kernel, initrd, title,
                        args=None, copydefault=True, makedefault=False):
        '''
        Add a boot entry with the given parameters.
        If copydefault is True, 'args' will be appended to the args from
        the current default entry.
        If makedefault is True, the new entry will be made the default.
        '''
        grubby_args = ["--add-kernel", kernel, "--title", title]
        if initrd:
            grubby_args += ["--initrd", initrd]
        if args:
            grubby_args += ["--args", args]
        if copydefault:
            grubby_args += ["--copy-default"]
        if makedefault:
            grubby_args += ["--make-default"]
        self._grubby(*grubby_args)


For German, the result is:

        linux   /vmlinuz-fedup root=/dev/mapper/vg_fedora-lv_root ro rd.md=0 rd.dm=0 SYSFONT=True rd.lvm.lv=vg_fedora/lv_swap rd.luks.uuid=luks-ba1042c9-4e46-4a24-a43e-0fc2b8e5bd3f rd.lvm.lv=vg_fedora/lv_root LANG=de_DE.UTF-8  KEYTABLE=de-latin1-nodeadkeys rhgb quiet upgrade systemd.unit=system-upgrade.target plymouth.splash=fedup

Dracut then reports that the commands LANG, KEYTABLE and SYSFONT are deprecated; the hdd encryption uses the US layout.

The script should replace those commands using locale, vconsole.font and vconsole.keymap. I'm not a python programmer and don't know how if I can fix the script, but I am sure someone else can. 

If you change the grub entry accordingly, hdd decryption uses the expected keyboard layout.

One other thing I've noticed: Dracut reports: 
dracut-cmdline[48]: /etc/locale.conf: line1: locale.LANG=de_DE.UTF-8: command not found

Not sure if this has some influence on the keyboard problem with X11/gdm?

I already knew all that, actually. Sorry, I thought this was common knowledge.

However, 'deprecated' has a standard meaning, and it's not 'doesn't work any more'. 'Deprecated' means 'still works, but is planned to stop working in the future, and you should switch over to the new shiny'. That's the point of a deprecation warning.

I actually tried to test this outside the context of encrypted passphrase entry, and seemed to get the result I expected. It's quite easy: pass 'KEYTABLE=de-latin1-nodeadkeys' (or whatever console layout you like) and also 'rd.break'. That'll dump you in a dracut rescue shell. For me, the specified keyboard layout is used in the rescue shell.

I couldn't confirm my result for actual passphrase entry, though, because I kept hitting other problems in fedup which obscured the test.

Hi, how do yo get the rescue shell? Is it a boot parameter (like 1?)

I have probably got this issue by upgrading from F18 using fedup.
I was not able to decrypt my LVM after the reboot once upgraded, but I were using the Fedora 19 rescue grub entry.
There is a change between those two entries. I just can't use an other keyboard layout.. I don't know my passphrase, just the blind scheme... :)

"and also 'rd.break'. That'll dump you in a dracut rescue shell"

and can't you type your passphrase in another OS or on any other system with the same keyboard layout to find out what it is?

First of all, rd.break just help getting the text to debug, but it just ask me my passphrase, I can't write anything else (than "stars" for passphrase).

Second, my passphrase is written using a dvorak(fr) scheme over a qwerty software layout, using an azerty printed keyboard :) No, I would prefer to reset (or add a new) the passphrase than trying all combinations.

Last.. Using keytable or keymap (=us, or =fr, or others as I don't have the right syntax) does not change anything. Using the rescue shell, even with this argument I can unlock my LVM. Which tells me that I use this argument in the wrong way.. It does not depend on keymap=fr nor keymap=us...

Note that KEYTABLE is capitalized in the stuff I wrote above. KEYTABLE= , not keytable= . Presumably that's significant. I don't remember specifically, but case is usually important.

Is there any documentation as to how to fix this (Where is the fine manual I should be reading)?  I tried changing KEYTABLE=dvorak to vconsole.keymap=dvorak in /boot/grup2/grub.cfg and I still get US in the encryption password.

The 21 century command line commands to set hostname/locale/timezone ( the most commonly used settings ) are

hostnamectl set-hostname <my hostname>
localectl set-locale <my locale>
timedatectl set-timezone <my timezone>
( man for details on all of those >

So "localectl set-keymap dvorak" should work for you ( you might need to add  'KEYMAP="dvorak"' to your '/etc/vconsole.conf' ) if not that's something we need to look into

Oh and note I think I came a across a dvorak bug somewhere which described itself as switching to querty if you changed vt's and if I recall correctly it was supposed to be fixed in recent kernels so if you experience it try the latest kernel from koji before filing a bug

johann: does that set the layout for dracut? I thought it only does it for VTs and X11.

(In reply to Adam Williamson from comment #33)
> johann: does that set the layout for dracut? I thought it only does it for
> VTs and X11.

If you rebuild the initramfs ( dracut -f ) I think it should contain the changes made to /etc/vconsole.conf ( KEYMAP="dvorak" )

he can simply check if it does via "lsinitrd -f etc/vconsole.conf" once he has rebuild the initramfs. If it does not that's something for Harald to deal with ;)

(In reply to thomas.michel from comment #24)
> One other thing I've noticed: Dracut reports: 
> dracut-cmdline[48]: /etc/locale.conf: line1: locale.LANG=de_DE.UTF-8:

err... "locale.LANG" is not a valid contents of locale.conf.

$ cat /etc/locale.conf
LANG=de_DE.UTF-8
LC_MESSAGES=en_US.UTF-8

(In reply to Adam Williamson from comment #33)
> johann: does that set the layout for dracut? I thought it only does it for
> VTs and X11.

If nothing is specified on the kernel command line, dracut/systemd-vconsole-setup defaults to /etc/vconsole.conf in the initramfs, which was copied on initramfs creation time.

Another thing: with F19, the initramfs only contains the keymaps and fonts, which were configured in /etc/vconsole.conf. The rescue/generic initramfs contains _all_ keymaps and fonts and can be fully set by the kernel command line params.

KEYTABLE=fr works (but not =us, as expected in fact) for the rescue entry.

Since Harald told about keymaps in initramfs included from /etc/vconsole.conf I thought the issue could be the missing layout.

However, my default /etc/vconsole.conf has just KEYMAP="fr". Therefor I should not change anything (KEYMAP Vs KEYTABLE?).
I tried to rebuild the initramfs (dracut -f), still no luck.
I tried other codes like KEYMAP=fr-latin1 but same issue.

And I still don't see any feedback about this option during the boot process (rd.break just gave me the dracut shell once failed to set my passphrase (tried with KEYTABLE=us) but ONLY with the rescue entry..)


$ localectl
   System Locale: LANG=fr_FR.UTF-8
       VC Keymap: fr-latin1
      X11 Layout: n/a

Alright, I were mad using the rescue Kernel then I just updated my passphrase.. Hopefully this bug would resolved for F20

https://bugzilla.redhat.com/show_bug.cgi?id=917718 is a duplicate of this bug.

This bug is still present for fedup-0.7.3-6.fc19, I just experienced the same problem when updating my F19 to F20 alpha. This bug is really annoying if you are from a country with non-QWERTY layout (e.g. Germany (de) using QWERTZ).

Fedup should simply use the previously default keyboard layout or at least warn the user that the layout is qwerty.

Proposed as a Blocker for 20-beta by Fedora user shaiton using the blocker tracking app because:

 This bug prevent user to reboot its system after FedUp if he does not have standard keyboard layout and is using LUKS. From last comment, this bug is still present.

This was already discussed as a blocker and rejected for a previous release, so it's likely that'd happen again, but we can always re-discuss it.

Sorry, I was not sure about this and has not found the blocker list (and finally reached the qa app).

This is a really anoying issue but probably not the worse.

(In reply to Kevin Raymond from comment #42)
> Sorry, I was not sure about this and has not found the blocker list (and
> finally reached the qa app).
> 
> This is a really anoying issue but probably not the worse.

Kevin, I disagree. I've encountered the problem now for the third time and was surprised that it is still not fixed. For users using characters that do not exist on the US layout (like German Umlaute ("ü")), there is no obvious way to upgrade using fedup. Especially as there is no warning about this. You would need to add a second encryption password before starting fedup, not somethink you would easily think off.

I still think it's a blocker.

Regards,

Tom.

This is still valid for F19->F20 upgrades, for the record. We can document it on CommonBugs, but it might make more sense to note it on the Upgrading and/or Fedup pages (too?)

whoops. never got discussed as a blocker because the RejectedBlocker whiteboard was still set (that looks like a bug in the blocker nomination app, will file; not sure if it's a blocker bug...;>). I expect this'll be rejected again, but it seems fair to honor the nominator's wishes, so marking as a proposed final blocker for tomorrow's review meeting.

thomas: FWIW i'd say it's probably 'smart computing' to only ever use ASCII characters in really crucial stuff like usernames, passwords, and encryption passphrase and so on. you can usually rely on ASCII input working somehow or other. anything beyond that is always somewhat more likely to mess up. this is kinda baked into computing at this point :/

we wound up fiddling with keymap config stuff quite a lot in f20, so we may want to reset and check this out from the ground up when upgrading from f20 to f21. be nice to finally squelch it. likely too late for f20.

This need to be fixed during this release cycle ( and what apparently needs to be done blocked to do so ) since this has been ongoing issue since F17.

yeah, it's definitely a 'we really ought to fix this now' bug.

So, I think I know what the actual problem is here. initramfs-fedup.img does not contain any keyboard layouts.

I did a clean install of F19 with only a UK English keymap. After doing that, /etc/vconsole.conf states 'KEYMAP="uk"' and the grub2.cfg for the kernel has "vconsole.keymap=uk" . initramfs for the installed kernel contains all keyboard layouts (seems to be a generic initramfs, not stripped). initramfs does *not* contain /etc/vconsole.conf, which is interesting, but doesn't break anything: on boot of the installed system, UK keymap is used for entering the passphrase (dracut gets it from cmdline, I guess).

I then yum updated to latest F19. After doing that, /etc/vconsole.conf is unchanged, new kernel has "vconsole.keymap=uk" in its cmdline, initramfs for the new kernel now contains only the UK keymap - so it's been stripped, but correctly - and initramfs for the new kernel now contains /etc/vconsole.conf . Rebooting with the new kernel, it behaves correctly (UK keymap used to enter passphrase and once booted).

I then ran fedup. After running fedup stage1 but not rebooting, the fedup entry in grub2.cfg has "vconsole.keymap=uk", but the fedup initramfs does not contain /etc/vconsole.conf . But more importantly, initramfs-fedup.img *doesn't contain any keyboard layouts at all*. Doing an lsinitrd on it and grepping for 'kbd' comes up empty. I checked this several times and double-checked the other two kernels to make sure I wasn't doing it wrong: they both show some stuff in /usr/lib/kbd, initramfs-fedup.img has nothing at all.

So I think that's the problem: for whatever reason, fedup's initramfs does not have the UK layout. dracut probably reads vconsole.keymap from cmdline and tries to load the UK layout, but it'll fail if it's not _there_.

So not surprisingly, after booting to fedup stage2, the bug occurs: I have to use US keymap to enter the passphrase.

oh - test used latest fedup available in F19 repos.

Discussed in 2013-12-04 Blocker Review Meeting [1]. Voted as an AcceptedBlocker. With this bug people with non-US keyboard won't be able to upgrade. Violates beta release criterion: "For each one of the release-blocking package sets, it must be possible to successfully complete an upgrade from a fully updated installation of the previous stable Fedora release with that package set installed. "

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2013-12-04/
[2] https://fedoraproject.org/wiki/Fedora_20_Beta_Release_Criteria#Upgrade_requirements

Once again the problem is that dracut is no longer actually capable of building a generic initramfs. Great.

dracut was designed to create a "generic" initramfs that could boot any system. fedup depends on that ability to boot the system to run the upgrade. In the cases where that works correctly, fedup works fine. When it doesn't (like here), fedup can't start the system correctly.

There's three ways to fix this:


1) Fix generic initramfs: rework the dracut 10i18n module so it actually includes *all* kbd/font stuff unless we're in hostonly mode. Then this'd just work.

2) Make dracut's hostonly logic available to fedup (et al): pull out the bits of 10i18n/module-setup.sh that grab the correct keymap, config file, and fonts; make it available to external programs. (dracut does this with commandline arguments already; see 'dracut --print-cmdline' in F20 and later.)

3) "Fix" it in fedup: replicate dracut's hostonly logic for the i18n module inside of fedup. Rewrite it in python, and maintain that code in parallel with dracut. (This is what fedup was starting to do with fedup.boot.need_mdadmconf() and fedup.sysprep.prep_boot(), but it quickly


Note that this is the exact same root cause as all our fedup problems with mdraid, dmraid, crypt, some LVM setups, etc. And we'll have these same problems with other dracut modules that don't properly handle generic vs. hostonly, like multipath, zfcp, and dasd.

We tried 3) for mdadm bugs (see https://github.com/wgwoods/fedup/commit/f937c4a, which fixed bug #895805) but it's quickly become apparent that this was going to be a maintenance nightmare. The long-term solution is definitely not to paper over the problem in fedup.

So, sure, count me as 100% on the 'fix it in initramfs generation' team. But, if I do this from a clean F20 VM:

dracut --no-hostonly test.img 3.11.10-300.fc20.x86_64

the test.img that's generated has everything you'd expect from /usr/lib/kbd in it. So we (dgilmore and I, who are trying to figure this out ATM) are not so sure it's a case of "dracut is no longer actually capable of building a generic initramfs", but a case of "dracut is leaving stuff out of the generic initramfs when we build it inside lorax for some reason". I've noted that anaconda's 'initramfs.img' is also missing the whole of /usr/lib/kbd - that just doesn't happen to cause us any major problems we know about (or *I* know about, anyway).

Right now our suspect for this specific issue is this check in dracut's modules.d/10i18n/module-setup.sh:

    checks() {
        for kbddir in ${kbddir} /usr/lib/kbd /lib/kbd /usr/share /usr/share/kbd
        do
            [[ -d "${kbddir}" ]] && \
                for dir in ${KBDSUBDIRS//,/ }
            do
                [[ -d "${kbddir}/${dir}" ]] && continue
                false
            done && break
            kbddir=''
        done

        [[ -f $I18N_CONF && -f $VCONFIG_CONF ]] || \
            [[ ! ${hostonly} || ${i18n_vars} ]] || {
            derror 'i18n_vars not set!  Please set up i18n_vars in ' \
                'configuration file.'
        }
        return 0
    }

if that fails, it just skips keyboard layout installation entirely. So we're currently suspecting that may be failing.

Presumably, if we're on the right track, there will turn out to be similar issues for all the other bits that are getting left out too. Which will be fun to track down.

We're still poking stuff, though. We may be wrong.

So, didn't figure out the answer yet, but got some possibly-interesting data. I compared three initramfs'es which really oughtn't to be all that different, and they kinda are.

Compared to a test generic initramfs built with "dracut --no-hostonly test.img 3.11.10-300.fc20.x86_64" in an F20 VM, both the 'rescue' initramfs from /boot on the same clean F20 install and fedup's initramfs are missing the following bits:

1. /usr/bin/gzip
2. everything that ought to have been installed by the 'i18n' dracut module: /usr/bin/stty
/usr/bin/setfont
/usr/bin/loadkeys
/usr/bin/kbd_mode
everything under /usr/lib/kbd

fedup's initramfs is also missing the dracut module 'virtfs', for reasons that are currently unclear to me. It's not just missing everything that ought to have been installed by that module - that module isn't in the list at the top of lsinitrd's output. That's different from 'i18n', which *is* in the list, but doesn't install anything. I can't find anything in fedup or fedup-dracut to account for this right now.

The lack of gzip in the initramfs could plausibly explain various other bugs, I guess. And I think as long as it's missing, loading compressed kbd layouts won't work even if we get the layouts themselves back in.

Well hey! I think I have it. At the top of /usr/lib/dracut/modules.d/10i18n/module-setup.sh there is this gem:

--------------------

# called by dracut
install() {
    if dracut_module_included "systemd"; then
        [[ -f /etc/vconsole.conf ]] || return 0
        unset FONT
        unset KEYMAP
        . /etc/vconsole.conf
        # if vconsole.conf has no settings, do not include anything
        [[ $FONT ]] || [[ $KEYMAP ]] || return 0
    fi

--------------------

I am pretty sure the effect of that is that the whole of the i18n module is just abruptly aborted if /etc/vconsole.conf doesn't exist - *even if we're building a generic initramfs*. I don't think that's right.

I tested this theory; in my clean F20 VM I renamed /etc/vconsole.conf to /etc/vconsole.foo and re-ran my test 'generic' initramfs build. Sure enough, it was now missing all the i18n stuff. Then I commented out the check above entirely, re-ran my 'generic' initramfs build, and all the i18n stuff showed up again.

dgilmore and I are talking it over and we think that's just completely bogus code. If you're not sure what's going on, we think the safe default would be to run install_all_kbd, not *not install anything at all*. The commit msg on the commit is not particularly reassuring:

https://git.kernel.org/cgit/boot/dracut/dracut.git/commit/modules.d/10i18n/module-setup.sh?id=f734347ae7da6943342b2a4e7be99b3e2c6feda7

"if /etc/vconsole.conf exists and is empty, then do not install anything."

One, that is not what that code does: it also doesn't install anything if the file doesn't exist at all, which is rather different from the claim. Two, it provides no explanation as to why this is a good idea in the first place.

At *minimum*, the code ought to do what it claims to do (it should only bail if the file exists and is empty, not if it doesn't exist at all). It also should probably be conditional on hostonly mode. Or it should just die.

I built an upgrade.img as close as possible to the official build (which dracut's habit of *completely silently* skipping modules it would otherwise have included if certain checks fail does NOT help with) but with the kbd bug avoided (it also included gzip, somehow; that'll need more investigation), copied it over initramfs-fedup.img on my upgrade test box, booted to fedup stage 2, entered the passphrase with UK keymap, and it worked. upgrade is running now.

BTW, this is going to break again for F20 to F21, I think. I've filed the reason why as https://bugzilla.redhat.com/show_bug.cgi?id=1038413 .

harald: I'd also note that the code in question, and indeed the whole of the i18n module, takes no account of the possibility that a non-US layout may be set on the cmdline rather than in vconsole.conf .

(In reply to Adam Williamson from comment #59)
> harald: I'd also note that the code in question, and indeed the whole of the
> i18n module, takes no account of the possibility that a non-US layout may be
> set on the cmdline rather than in vconsole.conf .

So, what we can do is:
- always include every keymap (even in hostonly mode)
- copy /etc/vconsole.conf and /etc/locale.conf in the initramfs (even in generic mode)
- anaconda should not add "vconsole.*" or "locale.*" to the kernel command line

If someone wants to change the layout, he could do it via:
"rd.vconsole.*=... rd.locale.*=" on the kernel command line, which only affects the initramfs behaviour, so changes with "localectl" on the real system are still honored.

The only problems I see is, if /etc/vconsole.conf is not present on the system or not configured correctly at the time of the initramfs image creation. So, we have to make sure, that anaconda writes /etc/vconsole.conf and /etc/locale.conf in the install root prior to the rpm transaction, or, what would be a waste, rerun dracut after it wrote /etc/{vconsole,locale}.conf.

I don't know how the upgrade tools should handle the old kernel command line parameters. Ideally they would be removed or at least converted to "rd.vconsole.*=... rd.locale.*=...".

rd.vconsole.keymap       = vconsole.keymap       or KEYMAP      or KEYTABLE
rd.vconsole.font         = vconsole.font         or FONT        or SYSFONT
rd.vconsole.font.map     = vconsole.font.map     or FONT_MAP    or CONTRANS
rd.vconsole.font.unimap  = vconsole.font.unimap  or FONT_UNIMAP or UNIMAP
rd.vconsole.font.unicode = vconsole.font.unicode or UNICODE   
rd.vconsole.keymap.ext   = vconsole.keymap.ext   or EXT_KEYMAP

rd.locale.LANG   = locale.LANG   or LANG
rd.locale.LC_ALL = locale.LC_ALL or LC_ALL

To take the most important point first:

"The only problems I see is, if /etc/vconsole.conf is not present on the system or not configured correctly at the time of the initramfs image creation."

Yes.

"So, we have to make sure, that anaconda writes /etc/vconsole.conf and /etc/locale.conf in the install root prior to the rpm transaction,"

No. You're only thinking about systems building their own initramfs'es again. This bug is about one of the many cases where we build generic initramfs'es. The initramfs in question is being built inside a chroot on a virtual machine which probably doesn't even have a flipping keyboard, nor has anaconda been run on it in living memory. You can't say 'dracut doesn't work right if vconsole.conf doesn't exist so the rest of the world should make sure it's only ever used if vconsole.conf exists', that's fairly absurd.

Think about the whole purpose of 'no-hostonly' mode. It is to build a *generic* initramfs, i.e., one that ought to be as independent as possible of the host on which it was built. Running checks on the configuration of the host machine when building a generic initramfs is entirely contrary to the concept of 'building a generic initramfs' in the first place. It simply makes no logical sense at all for *any* check like this to happen in no-hostonly mode.

What we should do instead is either this:

-    if dracut_module_included "systemd"; then
+    if [[ ${hostonly} ]]
+      then
+      if dracut_module_included "systemd"; then
         [[ -f /etc/vconsole.conf ]] || return 0
         unset FONT
         unset KEYMAP
         . /etc/vconsole.conf
         # if vconsole.conf has no settings, do not include anything
         [[ $FONT ]] || [[ $KEYMAP ]] || return 0
+      fi

(thanks dgilmore)

Or, alternatively, this:

-    if dracut_module_included "systemd"; then
-    if [[ ${hostonly} ]]
-      then
-      if dracut_module_included "systemd"; then
-        [[ -f /etc/vconsole.conf ]] || return 0
-        unset FONT
-        unset KEYMAP
-        . /etc/vconsole.conf
-        # if vconsole.conf has no settings, do not include anything
-        [[ $FONT ]] || [[ $KEYMAP ]] || return 0
-      fi
-    fi

since you still haven't explained why you think this 'check' is a good idea in the *first* place - saving a rather small amount of space?

If that's the goal then fine, keep the check, but do not run it in 'no-hostonly' mode, and make it consider the cmdline; if you can't make your 'optimization' correct don't make it at all.

To take the second point second, as this is only a sideline:

"- anaconda should not add "vconsole.*" or "locale.*" to the kernel command line"

We already changed that, but you're missing the point. It's a valid configuration method for the system that you are not taking into account. It doesn't matter whether anaconda is writing it or not; a system on which dracut is being used might be. You're writing a generic tool, not something that only works on systems written by anaconda then never touched again by the hand of Man. The code makes the assumption that it only needs to worry about ensuring i18n configuration is handled if /etc/vconsole.conf exists; that is an incorrect assumption.

dracut-034-62.git20131205.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/dracut-034-62.git20131205.fc20

dracut-034-64.git20131205.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/dracut-034-64.git20131205.fc20

Package dracut-034-64.git20131205.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dracut-034-64.git20131205.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-22831/dracut-034-64.git20131205.fc20
then log in and leave karma (feedback).

Fix confirmed with the repo at https://dl.fedoraproject.org/pub/alt/stage/scratch/20-Test-1 , which is built with the fixed dracut.

dracut-034-64.git20131205.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

(In reply to Adam Williamson from comment #61)
[snip]
> "- anaconda should not add "vconsole.*" or "locale.*" to the kernel command
> line"
> 
> We already changed that, but you're missing the point. [...]

new-kernel-pkg adds LANG= to the boot options, see bug 1010454

Doesn't do anything to keyboard layout.

Sounds like a bug in new-kernel-pkg then.

If "anaconda should not add "vconsole.*" or "locale.*"", neither should new-kernel-pkg. I think bug 1010454 can be reopened.

I'm trying to use fedup to upgrade from F20 (with updates) to rawhide. This bug seems to have resurfaced. I'm unable to get past the encryption passphase prompt, and when I Ctrl+Alt+F2 to switch consoles, text I type is entered in the US English keyboard layout (instead of the Dvorak layout I selected when I installed Fedora).

I'm sorry, this bug is different. This bug report is for broken keyboard layout AFTER upgrade; mine is that a broken keyboard layout PREVENTS the upgrade.

michael: see https://bugzilla.redhat.com/show_bug.cgi?id=881624#c58 , https://bugzilla.redhat.com/show_bug.cgi?id=1038413 - is that what you're seeing?

I think so, yes.

It should be pretty easy to check - see if the keymap is being passed in on the cmdline for the "upgrade" bootloader menu entry. If not, try adding it. If adding it works...then I'd say you're hitting 1038413 :)