Bug 981805

Summary: localectl / localed should use matching kbd layout for xkb layout when available
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: systemdAssignee: Zbigniew Jędrzejewski-Szmek <zbyszek>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: 20CC: gryt2, johannbg, kay, lnykryn, massi.ergosum, mfabian, msekleta, notting, petersen, plautrba, redhat-bugzilla, systemd-maint, vpavlin, vpodzime, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: systemd-208-6.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-24 23:48:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1028207    
Attachments:
Description Flags
systemd patch to prefer converted keymaps to legacy ones none

Description Adam Williamson 2013-07-05 21:25:34 UTC
In Rawhide (what's going to be F20), we now have a set of kbd keymaps which matches the full set (more or less) of xkb keymaps, thanks to https://bugzilla.redhat.com/show_bug.cgi?id=837292 . However, it looks like localectl still tries to convert to the 'best available' layout via /usr/share/systemd/kbd-model-map - i.e. it's kind of 'too clever', now.

[root@adam adamw]# localectl set-x11-keymap fr pc105 oss
[root@adam adamw]# localectl
   System Locale: LANG=en_US.utf8
                  LC_NUMERIC=en_CA.utf8
                  LC_TIME=en_CA.utf8
                  LC_MONETARY=en_CA.utf8
                  LC_MEASUREMENT=en_CA.utf8
       VC Keymap: fr
      X11 Layout: fr
       X11 Model: pc105
     X11 Variant: oss

In this case, for instance, I'd expect VC keymap to be 'fr-oss', not 'fr'. That layout does exist:

[root@adam adamw]# ls /usr/lib/kbd/keymaps/xkb/fr-oss.map.gz 
/usr/lib/kbd/keymaps/xkb/fr-oss.map.gz

and can happily be loaded with 'loadkeys', and works.

I don't know if we want to keep kbd-model-map at all, but we should certainly use the 'exact match' for an xkb layout so long as it exists.

Comment 1 Kay Sievers 2013-07-05 21:42:10 UTC
I'm convinced that we should rip out all this magic and handle only the
xkb keymaps.

Any possibly needed compat, we should provide by symlinks, not by
the static list we ship currently.

Comment 2 Adam Williamson 2013-07-05 21:45:53 UTC
That would certainly be the simplest plan. How would we handle upgrades?

Comment 3 Adam Williamson 2013-07-05 21:46:29 UTC
(I haven't checked if there are actually any keymap names that exist in the 'old kbd' set but not the 'new xkb' set, but it seems possible).

Comment 4 Vratislav Podzimek 2013-07-08 09:09:12 UTC
I just want to mention that resolving this bug would help Anaconda a lot.

Comment 5 Vratislav Podzimek 2013-09-06 09:20:34 UTC
Kay, any update on this?

Comment 6 Fedora End Of Life 2013-09-16 14:24:19 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 20 development cycle.
Changing version to '20'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora20

Comment 7 Adam Williamson 2013-09-17 22:51:12 UTC
systemd folks, we would really appreciate if this can be addressed ASAP. It already looks like it'll miss F20 Alpha. We really need to have the 'new' keyboard handling hooked up as it's intended to work by Beta, so we can test it and not be trying to write and test it on the fly for final.

Comment 8 stan 2013-09-21 01:16:54 UTC
This bug seems like a good place to mention a problem I'm having with the new keyboard process in F20.

I run a unique custom keymap (ergonomic).  In the past, I've just copied the two keymap files (same keymap, one for console, one for X) into the appropriate places, changed the appropriate configuration files, and was good to go.  Tweak once at install, then never have to touch anything again.

It was pretty easy to tweak X to accept the keymap in F20, a return to the days when the evdev.xml and base.xml files were used.  Put the custom keymap file in xkb/symbols and a stanza for it in the xml files in xkb/rules, and in business.  I made it default by using a file in /usr/share/X11/xorg.conf.d, though I usually use a .Xkbmap file in my home directory.  Unfortunately, since I am unable to officially install F20 because of an obscure bug in the image, I used unconventional manual means, and can't login for now.  But no matter what I do, I can't get systemd to accept my custom console file.  I put the name in /etc/vconsole.conf (the replacement for /etc/sysconfig/keyboard), and it ignores it.  I suspect that it isn't actually reading the /lib/kbd/keymaps/xkb directory to see which keymaps are present, but is instead looking at a fixed internal list derived from approved X keymappings.  And since my custom keymap isn't on that approved list, it's ignored and systemd defaults to us.

Would it be possible to have systemd look at the keymaps present for the console, so people who want to run unique custom keymaps can do so?  Or maybe, it could be added to a conf file somewhere.  Or perhaps there is already a way to do this, and I'm not just aware of it yet.  If so, please tell me.

Thanks.

Comment 9 Adam Williamson 2013-09-21 01:26:53 UTC
"This bug seems like a good place to mention a problem I'm having with the new keyboard process in F20."

No, it isn't. One bug report per bug. Please file yours separately.

Comment 10 stan 2013-09-21 03:00:24 UTC
Do you consider my problem a bug?  I thought of it more as a possible flaw in the design.  The process hasn't even been finalized yet.  :-)  

I filed a bug against systemd.

https://bugzilla.redhat.com/show_bug.cgi?id=1010516

Comment 11 Adam Williamson 2013-11-17 01:27:31 UTC
This still seems to be broken in current F20: when you select UK English keyboard layout, vconsole.conf is written as:

KEYMAP="uk"

which is the old, kbd "UK" layout. The new, xkb one would be "gb". This implies that localed is still 'converting' xkb layout names to kbd ones, whereas we don't want it to any more.

Comment 12 Zbigniew Jędrzejewski-Szmek 2013-11-18 18:58:29 UTC
Created attachment 825795 [details]
systemd patch to prefer converted keymaps to legacy ones

This patch should fix the cases described in the bug, X11's fr-oss will be mapped to fr-oss, gb to gb. The patch is not complete, because the mapping the other way is asymmetrical now, but maybe that's not so important now.

It would be great if somebody could look over the general logic of the change.

Comment 13 Zbigniew Jędrzejewski-Szmek 2013-11-18 19:01:53 UTC
(In reply to Kay Sievers from comment #1)
> I'm convinced that we should rip out all this magic and handle only the
> xkb keymaps.
> 
> Any possibly needed compat, we should provide by symlinks, not by
> the static list we ship currently.
I think it's too late before F20 to do major changes. Let's just fix stuff, and then let's remove the legacy logic from systemd totally. Removing it totally would mean that *setting* keymaps through legacy names would stop working. But *using* previously configured keymaps with legacy names should continue to work.

Comment 14 Adam Williamson 2013-11-18 20:07:42 UTC
That sounds like a sensible change so far as it goes. Honestly, the issue that worries me the most is https://bugzilla.redhat.com/show_bug.cgi?id=837292#c46 , though - and I can't think of an immediately good way to deal with that that would be safe for the short horizon we have before F20. Especially if, as vitezslav seems to suggest, it turns out there actually isn't any way to cause the 'legacy' layouts to be loaded if they're not in the loadkeys path.

Comment 15 Zbigniew Jędrzejewski-Szmek 2013-11-18 20:11:54 UTC
(In reply to Adam Williamson from comment #14)
> That sounds like a sensible change so far as it goes. Honestly, the issue
> that worries me the most is
> https://bugzilla.redhat.com/show_bug.cgi?id=837292#c46 ,
Exactly. That's why removing 'legacy' layouts isn't much of an option. I sincerely doubt that anyone's going to come up with a solution... Let's just wait a bit more until the console is obsolete for everything except being used as a debug tool.

Comment 16 Zbigniew Jędrzejewski-Szmek 2013-11-19 01:52:40 UTC
Patch applied in http://cgit.freedesktop.org/systemd/systemd/commit/?id=0732ef7. I'll be putting out an update soon.

Comment 17 Adam Williamson 2013-11-19 19:30:08 UTC
For the record, filed https://bugzilla.redhat.com/show_bug.cgi?id=1031848 for the switchable layout problem.

For F20, I think the best gameplan would be to get this patch and the kbd change from https://bugzilla.redhat.com/show_bug.cgi?id=1028207 both built and included ASAP, come up with some kind of ugly hack for the layouts we know should use legacy switchable kbd layouts not xkb-converted ones, and for the longer-term, just focus on getting kmscon into Fedora.

Comment 18 Adam Williamson 2013-11-20 04:31:34 UTC
I did a test live image build with systemd-208-5.fc21, but the fix for this either wasn't correctly included or didn't work. 'localectl set-x11-keymap gb' still gives me 'uk' as the 'VC Keymap' and in /etc/vconsole.conf, not 'gb'.

(the systemd-208-5.fc21 build 'failed', but both x86-64 and i686 succeeded; it was only ARM that failed. I was able to get the x86_64 packages out of the koji task.)

Comment 19 Zbigniew Jędrzejewski-Szmek 2013-11-20 04:35:13 UTC
> I did a test live image build with systemd-208-5.fc21
Sorry, I screwed up and included only patches 1-99, missing 100-117 :)
Then I too was confused why it's not working. systemd-208-6.fc21 or systemd-208-6.fc20 should be fine. They are basically the same, except that systemd-208-6.fc20 has the dependency on kmod >= 15 reverted.

Comment 20 Fedora Update System 2013-11-20 04:54:28 UTC
systemd-208-6.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/systemd-208-6.fc20

Comment 21 Adam Williamson 2013-11-20 05:03:59 UTC
aha, thanks! I'll rebuild my live image and re-test.

Comment 22 Adam Williamson 2013-11-20 05:07:08 UTC
btw, you could ask kay to push kmod 15 stable for f20: it's been in updates-testing forever and has +10 karma, but he didn't set an auto-stable-push karma threshold and has not pushed it manually.

Comment 23 Adam Williamson 2013-11-20 05:22:19 UTC
remark on the fix, btw: I don't know if it's possible, but could localed ask xkb for its search path, rather than having what it hopes is the same search path coded in? (KBD_KEYMAP_DIRS)

Comment 24 Zbigniew Jędrzejewski-Szmek 2013-11-20 05:24:54 UTC
Yeah, that's a bit ugly. Maybe kbd could provide a pkgconfig file with the search path?

Comment 25 Adam Williamson 2013-11-20 05:57:14 UTC
OK, with 208-6, I confirm the fix, and with this combined with a kbd with vitezslav's and my fixes for other issues, overall behaviour looks sane.

Comment 26 Fedora Update System 2013-11-24 03:33:17 UTC
Package systemd-208-6.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-208-6.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-21935/systemd-208-6.fc20
then log in and leave karma (feedback).

Comment 27 Fedora Update System 2013-11-24 23:48:41 UTC
systemd-208-6.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.