Bug 1031848

I did an actual install test just to verify my reading of this issue is correct. It seems to be. If you install in Russian, then anaconda does something sane: it configures both Russian and English (US) keyboard layouts. In anaconda and then in X in the installed system, everything works great - both layouts are available everywhere, and you can switch between them. But at the console in the installed system, you have the xkb-converted 'ru' layout loaded, and you cannot input anything but Cyrillic characters.

(In reply to Adam Williamson from comment #0)

> A more comprehensive fix, however it looked, would I think require langtable
> to know about these cases - cases where it is standard, expected practice
> that both 'native' and US layouts will be defined. Right now I don't see
> that langtable really expresses this. If we had that data in langtable, we
> might be able to do something ugly like have the script that generates kbd
> layouts from xkb layouts generate a combined native+us kbd layout for those
> cases?

langtable has this information. It can be queried
with the supports_ascii() function:

    def supports_ascii(keyboardId=None):
        '''
        Returns True if the keyboard layout with that id can be used to
        type ASCII, returns false if the keyboard layout can not be used
        to type ASCII or if typing ASCII with that keyboard layout is
        difficult.

        >>> supports_ascii("jp")
        True
        >>> supports_ascii("ru")
        False
        '''
        if keyboardId in _keyboards_db:
            return _keyboards_db[keyboardId].ascii
        return True

Anaconda uses this to offer US English layout as a second layout by
default during the installation if the layout for the chosen language
does not support ASCII. I.e. when the language chosen is Russian,
anaconda will offer both Russian and US English layout by default, if
the language chosen is Japanese, it will only offer Japanese layout by
default (because the Japanese layout can be used to input ASCII
without problems).

ahh, I saw that key but didn't catch what it was for. Cool. So we do have the info to try and do something with it at the kbd level, whatever that is.

I wouldn't put that logic into kbd...

I think that easiest way is this one - when during install when users selects one of switchable layout, appropriate keymap variant should be written to /etc/vconsole.conf for console by anaconda/systemd (or whatever is doing it now).

For example, for

cz-us-qwertz		cz,us

it is cz-lat2.

In case that kbd would be without current "old keymaps names to xkb converted keymaps" symlinks, correct keymap cz-lat2 from legacy directory will be selected and switching will work.

The only disadvantage I see is that cz-lat2 is probably not exactly same as cz-us-qwertz. But I don't think that this is problem, because virtual console users are used to having just these "old" keymaps.

Well, couple of problems. One, I don't think your example is right. The table I gave is the first two columns of kbd-model-map for affected layouts. The first entry is the kbd layout name, the second is the matching set of xkb layouts: so what "cz-us-qwertz     cz,us" means is "if the user picks cz-us-qwertz kbd layout, select the cz,us X11 layouts".

But more significantly, we have another problem with the approach: name collisions. In some cases the kbd and xkb names differ, but in at least three that I can see, they don't: ru, gr and by. If you pick Russian, Greek or...whatever the hell by is..., then how do we manage to get the legacy keymap loaded rather than the xkb-converted one? We can't do it by name: whether we're using symlinks or the path approach you just posted to https://bugzilla.redhat.com/show_bug.cgi?id=1028207 , the xkb-converted layout would 'win' as things stand, AFAICS.

So I guess the way to address that in the symlink approach would be to drop the symlinks for those three layouts and instead have the 'real' legacy layouts present in the /usr/lib/kbd/keymaps dir. The way to address it in the path approach would be to...simply not include the xkb-converted versions of those three layouts at all, I suppose? Then the legacy ones would be loaded...

One thing Kay points out is that this should become moot whenever we get around to switching to kmscon: http://www.freedesktop.org/wiki/Software/kmscon/ . It sounds like, to a rough approximation, in a kmscon world your console keyboard configuration is exactly the same as your xkb keyboard config - it uses libxkbcommon to pretty much implement Xkb entirely. As long as that includes layout switching, then once we have kmscon we can ditch all the kbd crap entirely and simply write an xkb configuration from anaconda and everything will work.

If we could, say, switch to kmscon for F21, then we'd really just need to try and implement whatever quick fixes and hacks we can safely do to make F20 work as decently as possible, without worrying about 'longer-term' fixes.

OK, so I think I have a good idea for the best way to hack this for F20.

Where kbd.spec actually does the xkb conversion, we should add a check against langtable and don't convert layouts which have ASCII support set as 'false'.

I think the result of that would be that, where kbd-model-map has a mapping to a 'good' legacy layout, that legacy layout would be used, and where it doesn't, a US layout would be used. Which I think is a sensible place to be.

It should be relatively trivial to add this logic for someone who knows what they're doing, I think - it should just be a single query against langtable-python. I don't know what I'm doing, so it'd take me longer :) But I can try.

This seems saner than hacking in a static list of 'rm -f's after the conversion occurs.

If we can't get it done 'nicely' via langtable we could also do this hack I just came up with:

zgrep -L "U+0041" * | xargs rm -f

run against the set of generated keymaps would delete any which do not contain a mapping for a capital "A", which seems a reasonable proxy for 'layouts that can't input ASCII'. Ugly, but probably effective.

http://koji.fedoraproject.org/koji/taskinfo?taskID=6203682 is a kbd built with the change from https://bugzilla.redhat.com/show_bug.cgi?id=1028207#c61 and *also* with the hack I just described added - all the converted layouts which fail that grep test are deleted right after the conversion runs. I'm planning to build a test live image with this kbd and the new systemd build that should be showing up soon, and see how that behaves.

OK, I built a live image with my kbd scratch build and https://admin.fedoraproject.org/updates/systemd-208-6.fc20 and did installs in Russian, Arabic, and UK English. Behaviour in all cases was as I'd expect: the Russian install loaded the 'legacy' Russian kbd layout and switching worked, the Arabic install wrote 'ar-digits' into vconsole.conf (indicating that localed is still 'converting' layout names when appropriate) but actually wound up with a US console layout because ar-digits doesn't actually *exist* (indicating that fallback mechanisms work as expected), and the UK install loaded the xkb-converted 'gb' layout, not the legacy 'uk' one. So unless anyone can see a flaw in my Awesome Plan somewhere, I'd suggest we just go with it. I'm actually pretty happy with the 'grep hack'; it's simple, and it seems to achieve what we need to achieve.

Some keyboard layout information added to langtable by Adam:

https://github.com/mike-fabian/langtable/commit/bc68d6a95a25f104ca54968253b5bde39c301e97

https://admin.fedoraproject.org/updates/langtable-0.0.20-1.fc20

Discussed at 2013-11-20 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-11-20/f20-blocker-review.2013-11-20-17.00.log.txt . This was accepted as a release blocker per Alpha criterion "A system installed without a graphical package set must boot to a state where it is possible to log in through at least one of the default virtual consoles." in the case of installing with an affected keyboard mapping - you won't be able to log in if you entered a username/password using Roman characters (which is very likely, people usually don't use native characters for those). It also likely affects unlocking of encrypted partitions.

The solution looks fine to me.

Adam, could you please attach the patch explicitly? I'm not able to grab it from the scratch build.

vitezslav: simply add these lines right above find_lang:

# wipe converted layouts which cannot input ASCII (#1031848)
zgrep -L "U+0041" $RPM_BUILD_ROOT/lib/kbd/keymaps/xkb/* | xargs rm -f

that's all it takes.

kbd-1.15.5-11.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/kbd-1.15.5-11.fc20

Package kbd-1.15.5-11.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kbd-1.15.5-11.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-22058/kbd-1.15.5-11.fc20
then log in and leave karma (feedback).

kbd-1.15.5-11.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.