Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1238683

Summary: localectl does not set a proper console font after switching keymap
Product: Red Hat Enterprise Linux 7 Reporter: Martin Kolman <mkolman>
Component: systemdAssignee: systemd-maint
Status: CLOSED WONTFIX QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.2CC: daherrma, mkolman, msekleta, systemd-maint-list, vpodzime
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-16 10:41:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martin Kolman 2015-07-02 12:15:04 UTC
Description of problem:
When localectl is used to switch keymap to one containing non-latin characters and locale is set one that outputs non-latin characters it results in a screen with a lot of white rectangles.


Version-Release number of selected component (if applicable):
systemd 219-4

How reproducible:
always

Steps to Reproduce:
1. install RHEL 7.2
2. localectl set-keymap jp
3. export LANG=ja_JP.utf8
4. date

Actual results:
Output from the date utility contains a lot of white rectangles. The same thing happens for any other command line utility that has japanese (or other non-latin characters) in its output.

For example:
https://m4rtink.fedorapeople.org/screenshots/rhel/console_locale_ja_jp_june30.png

If the output contains only latin characters it looks fine:
https://m4rtink.fedorapeople.org/screenshots/rhel/console_locale_en_us_june30.png

Expected results:
The localectl utility sets a font that can display the selected keymap, so that users see the expected non-latin characters instead of white squares.

Comment 2 Michal Sekletar 2015-07-02 13:32:58 UTC
systemd-localed doesn't set console font and it doesn't expose any API to do that. Console font is loaded at boot time by systemd-vconsole-setup.service. If you change your keymap such that current font doesn't support all the necessary glyphs then you need to change /etc/vconsole.conf config file where you change FONT= variable to appropriate font and reboot. Alternative (not sure whether that works reliably) you can call setfont manually.

Comment 3 Vratislav Podzimek 2015-07-03 12:40:15 UTC
(In reply to Michal Sekletar from comment #2)
> systemd-localed doesn't set console font and it doesn't expose any API to do
> that. Console font is loaded at boot time by systemd-vconsole-setup.service.
> If you change your keymap such that current font doesn't support all the
> necessary glyphs then you need to change /etc/vconsole.conf config file
> where you change FONT= variable to appropriate font and reboot. Alternative
> (not sure whether that works reliably) you can call setfont manually.

Yes, 'setfont' works. Actually, it's the only way to get the proper font without a reboot as /etc/vconsole.conf is only read on boot.

However, if there's a tool (and DBus service) that is supposed to provide the functionality for switching VConsole keymap, which localectl/localed is, it should do such switch correctly with everything that's needed. Otherwise it's just a wrapper around 'loadkeys' which is not really useful.

Comment 4 Michal Sekletar 2015-07-03 15:12:06 UTC
How is localed supposed to know which font contains glyphs needed to properly support given keymap. If there is some canonical source of such information then we could probably add this feature.

Comment 5 David Rheinsberg 2015-07-15 13:07:15 UTC
The linux-console only allows fonts with max 512 glyphs. 128-256 of them are required for ASCII compat and general purpose glyphs, leaving usually only 256 free-to-use glyphs. Many scripts (including Japanese) require far more glyphs to represent normal text. Furthermore, the normal glyph-reduction by using combining-characters is not supported by the linux console. Instead, each character is represented by one glyph.

If you set your locale to "ja_JP.utf8", you explicitly tell applications to use the full Unicode range of characters. However, your font clearly does not support that. Furthermore, there is no font that can support that range.

I don't know what you expect `localectl` to do here. There is clearly no correct choice for a font that can represent such extensive scripts on the linux console. Furthermore, I'm not aware of a way to map locales to fonts (which, btw, sounds backwards, anyway). I also disagree that `localectl` should do any black magic behind the users back.


But, clearly, there's an issue here: You're totally right to set a Japanese UTF-8 locale, and that works totally fine on extended terminals (like gnome-terminal). However, it clearly breaks the linux-console. I'm not aware of a real fix for this. My recommendation is to stop using the long broken and ancient linux-console and use proper terminal emulators instead. Alternatively, you can use an english locale for your linux-terminal logins. Last idea: disable UTF-8 on the linux-console, as it clearly is not capable of properly displaying its full range of codepoints.

Comment 6 Martin Kolman 2015-07-16 10:02:36 UTC
(In reply to David Herrmann from comment #5)
> The linux-console only allows fonts with max 512 glyphs. 128-256 of them are
> required for ASCII compat and general purpose glyphs, leaving usually only
> 256 free-to-use glyphs. Many scripts (including Japanese) require far more
> glyphs to represent normal text. Furthermore, the normal glyph-reduction by
> using combining-characters is not supported by the linux console. Instead,
> each character is represented by one glyph.
> 
> If you set your locale to "ja_JP.utf8", you explicitly tell applications to
> use the full Unicode range of characters. However, your font clearly does
> not support that. Furthermore, there is no font that can support that range.
Sure, that!s indeed an issue - I have been discussing this with Mike Fabian (he is among other things maintainer of the langtable project) and he confirms that the default Linux console indeed can't display Japanese (and Hindy and other) fonts at all. There is apparently an alternative console implementation called fbcon which can do that, but reportedly has some quality issues.

But on the other hand while it is impossible to display Japanese (& some other languages) in the default Linux console, it can still correctly displaying Cyril, Greek and maybe even Arabic fonts.

> 
> I don't know what you expect `localectl` to do here. There is clearly no
> correct choice for a font that can represent such extensive scripts on the
> linux console.
I would expect localectl to set a correct font in the console, if possible for example a Greek font for the Greek keymap. And for fonts that can't be displayed it could at least print a warning.

>Furthermore, I'm not aware of a way to map locales to fonts
> (which, btw, sounds backwards, anyway). 
Langtable provides this information[0] - it can map locales to font types. It can also list script types for the given locale/language - this can be used to detect if the font corresponding to the language can be displayed (it is one of 'Latn', 'Cyrl' or 'Grek') or not.

>I also disagree that `localectl`
> should do any black magic behind the users back.
> 
> 
> But, clearly, there's an issue here: You're totally right to set a Japanese
> UTF-8 locale, and that works totally fine on extended terminals (like
> gnome-terminal). However, it clearly breaks the linux-console. I'm not aware
> of a real fix for this. My recommendation is to stop using the long broken
> and ancient linux-console and use proper terminal emulators instead.
Sure, but that's not always one option - there are times there is no GUI running and only the console is available. And it is quite unfortunate if the user is greeted with a screen full of white rectangles at such time.
> Alternatively, you can use an english locale for your linux-terminal logins.
> Last idea: disable UTF-8 on the linux-console, as it clearly is not capable
> of properly displaying its full range of codepoints.

[0] https://github.com/mike-fabian/langtable/blob/master/data/languages.xml

Comment 7 Michal Sekletar 2015-07-16 10:41:07 UTC
We are not adding any code to partially hack around broken kernel text consoles. In future this will be fixed by console implementations in user-space. Until then, sorry.