This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1031754 - [RFE] Add manual information about locale installation, localedef, and locale source file format.
[RFE] Add manual information about locale installation, localedef, and locale...
Status: NEW
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: glibc (Show other bugs)
7.4
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: glibc team
qe-baseos-tools
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-18 12:15 EST by James B. Byrne
Modified: 2017-07-20 01:56 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Sourceware 18408 None None None Never

  None (edit)
Description James B. Byrne 2013-11-18 12:15:01 EST
Description of problem:

A common requirement for our systems is to standardize the date representation to the format yyyy-mmm-dd regardless of the language employed by the system administrators.  On RHEL6 this process is unbelievably complex and intricate given the trivial nature of the change.

Trying to set a custom locale in RHEL6 is exceptionally poorly supported.  A critical utility, localedef, has absolutely no documentation other than the --help and --usage options of the utility itself and no reference to its existence in the man pages. A search for locale in the man pages returns a plethora of PERL related pages but not a single reference to the system locale or localedef commands.  Even when one knows about the relationship between locale and glibc there exists no man page that refers to glibc either.

Even the name for the locale system settings file /etc/sysconfig/i18n seems to have been chosen to obscure rather than illuminate how one actually sets the system locale. (I understand that this particular issue is under consideration for change but it is nonetheless the present case).

Further adding to the complexity is the fact that editing a locale definition file is unsupported by any system utility that I could find.  One must manually determine what the literal change strings should look like, convert those changes into utf8 code-points, manually insert them into the correct location in a custom locale definition file, and only then install the changes by using localedef to generate the necessary support files.  The format of the locale definition file is itself undocumented in the RH man pages and alternative external references vary in their descriptions.  The most useful I have founds is http://man7.org/linux/man-pages/man5/locale.5.html but that is not authoritative insofar as I can determine.

Finally, the procedure required for the installation of a custom locale file is also undocumented and must be discovered by trial and error.

Version-Release number of selected component (if applicable):

glibc-common-2.12-1.107.el6_4.5.x86_64

How reproducible:

Attempt to change the locale date format in any standard locale definition or create a custom locale definition with a non-standard date format.

Steps to Reproduce:
1. look for locale in man pages using apropos
2. look for i18n in man pages using apropos
3. look for locale in redhat.com 
4. look for a locale customization utility

Actual results:

1. returns perl info
2. returns setsysfont info
3. identifies /etc/sysconfig/i18n as where the system locale is set
4. does not exist

Expected results:

1. returns a list of man pages respecting all glibc utilities and settings files together with explicit procedural references on how to employ and customize locale settings.
2. returns a list of all glibc calls sourcing /etc/sysconfi/i18n and references to related glibc utilities and commands.
3. returns a complete discussion of system locale default settings and user customization options together with procedural references.
4. Provision of a system utility that allows interactive creation, modification and installation of customized system and user locales.

Additional info:

Setting custom locale info for specific hosts and individual users should not be as obscure and difficult as is currently the case.  A system utility that allows user customization of commonly altered representations such as date, time, currency, and decimal numbers should be available to system administrators for system default configuration and to user accounts for configuration on a per user basis.

The current arrangement is clearly defective from a usability perspective.
Comment 1 Carlos O'Donell 2013-11-18 12:31:19 EST
(In reply to James B. Byrne from comment #0)
> The current arrangement is clearly defective from a usability perspective.

I fully agree that we don't do anything to help users install custom locales by themselves. We obviously do offer support for those customers who have subscriptions and pay for support.

It would be a very good thing if we did a better job of documenting the process of installing and keeping a locale installed. I think that should go into the glibc manual.

As for the documentation of localedef and the locales themselves, this isn't covered in any manual page because it's part of the POSIX standard.

POSIX Issue6 definition of `localedef'
http://pubs.opengroup.org/onlinepubs/009695399/utilities/localedef.html
- Also provided by `man localedef' but only if you have the POSIX Programmer's Manual Pages addition to the man-pages package.

POSIX ISsue6 definition of locale file format:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html#tag_07_03
- No manual page, see POSIX documentation directly.

Action items:
- Add a section in glibc's manual (info libc) under `Locales' to talk about how to install a locale permanently into your system so that it isn't lost upon upgrade.
- Add a reference to the POSIX standard when it comes to using localedef and the source locale file format.
- Add a section in the manual under `Locales' about the extra options for localedef which are GNU extensions.

Does that sound good?
Comment 3 James B. Byrne 2013-11-19 10:41:13 EST
Increased documentation certainly would be a great improvement over the present situation and I support that approach.  However, improving documentation alone still leaves unanswered the tedious difficulties facing anyone wishing to make what is nothing more than a cosmetic change to the representations of date, time, currency and numbers presented by system utilities like ls.  

I believe that the degree of user control given over these elements as matters of personal preference in the vast majority of end-user orientated software (the evident intransigence of the OpenOffice and LibreOffice development teams on the subject notwithstanding) indicates that burying the formatting of these elements in an obscure configuration file using a technique that is charitably described as arcane is not really 'state of the art'.

Is there no reasonably robust way imaginable that these four elements could be overridden at the host and user level by a more user-friendly supplemental configuration file (~/.locale  or /etc/locale) which employs standard printf and date %format specifiers?  Something using a hierarchy of /etc/sysconfig/i18n (why this is not called /etc/locale.conf I cannot fathom) overridden by some suitably named file such as /etc/local_locale.conf; or simply extend /etc/sysconfig/i18n to have additional key words to override specific LOCALE elements with custom formatting. And then have ~/.locale.conf contain any user customization that overrides the system level customization.

In any case improved documentation is a good start.
Comment 4 Carlos O'Donell 2013-11-19 12:37:42 EST
(In reply to James B. Byrne from comment #3)
> In any case improved documentation is a good start.

I'm glad we agree on this.

The other points you make are all valid, but much more difficult to implement in reality. A locale is not really designed to be edited by end-users but rather by a distribution. The locales were designed to map directly to geopolitical zones where governments defined language standards being used in their creation. The locale interface was not designed as a way for users to tweak the number representations for their systems. Thus you can see that given this set of design criteria it results a radically different implementation. One that is rich, and flexible, but difficult to use.

What you are describing is useful, but has a different set of design criteria and use cases. It would be a considerable project to do this, and because of that I'm not going to include that work in this bug. If you want such a feature you're going to need to file a bug upstream with glibc to get that functionality.

Retitling accordingly.
Comment 5 James B. Byrne 2013-11-19 14:19:10 EST
(In reply to Carlos O'Donell from comment #4)

> 
> The other points you make are all valid, but much more difficult to
> implement in reality. A locale is not really designed to be edited by
> end-users but rather by a distribution. The locales were designed to map
> directly to geopolitical zones where governments defined language standards
> being used in their creation. 

However, in Canada's case the standard government mandated representation of a date and time has been yyyy-mm-ddTHH:MM:SS (ISO8601) since 1989.  So, the present settings of the en_CA (and doubtless the en_FR as well) LOCALE as distributed in RHEL6 is considerably out of date.  How does one get this changed in the distribution as shipped? This is the very issue that led to my difficulties to begin with.  If there was a version of en_CA (en_CA@ISO8601) shipped with the officially recognized standard date and time formats set therein then that would more than suffice for my requirements.
Comment 6 James B. Byrne 2013-11-19 14:37:09 EST
ref:

CAN/CSA-Z234.4-89 (R2007)
All-Numeric Dates and Times

This National Standard of Canada is based on International Standard ISO 8601:1988.

1. Scope

1.1 This Standard covers the writing, typing, and printing of dates and times when all-numeric forms are used, and is applicable for general use.

1.2 This Standard is less comprehensive than, but not in conflict with, CAN/CSA-Z234.5 (Adopted ISO Standard 8601), Data Elements and Interchange Formats - Information Interchange - Representation of Dates and Times. However, see also Table 1.
Status: Withdrawn
SDO: CSA
Language: English
Publish date: 1989-12-31
Supersedes: CAN3-Z234.4-79
Reaffirmed: 2007-06-27
Keywords: STANDARDIZATION, TIME, DAYS, DATES (CALENDAR), NUMERIC REPRESENTATION, DATA REPRESENTATION, SECONDS (TIME), REPRESENTATION OF CHARACTERS, HOURS, MINUTES (TIME)
ICS Codes: 01.140.10;
Standard Number: CAN/CSA-Z234.4-89 (R2007)


This national standard has since been withdrawn in favour of strict ISO 8601 compliance which was adopted as standard in 1988:


ISO 8601:1988
Data elements and interchange formats -- Information interchange -- Representation of dates and times
Language: en
Circulation date: 1988-06-16
ICS Codes: 01.140.30;
Standard Number: ISO 8601:1988
Comment 7 Carlos O'Donell 2013-11-19 15:29:57 EST
(In reply to James B. Byrne from comment #5)
> (In reply to Carlos O'Donell from comment #4)
> 
> > 
> > The other points you make are all valid, but much more difficult to
> > implement in reality. A locale is not really designed to be edited by
> > end-users but rather by a distribution. The locales were designed to map
> > directly to geopolitical zones where governments defined language standards
> > being used in their creation. 
> 
> However, in Canada's case the standard government mandated representation of
> a date and time has been yyyy-mm-ddTHH:MM:SS (ISO8601) since 1989.  So, the
> present settings of the en_CA (and doubtless the en_FR as well) LOCALE as
> distributed in RHEL6 is considerably out of date.  How does one get this
> changed in the distribution as shipped? This is the very issue that led to
> my difficulties to begin with.  If there was a version of en_CA
> (en_CA@ISO8601) shipped with the officially recognized standard date and
> time formats set therein then that would more than suffice for my
> requirements.

It's a bug if the locale doesn't meet the government standards. File a bug against RHEL6 and it will go into the queue of bugs to get fixed. Please understand that the bug will be prioritized against other bugs to fix in the release.

You should also file an upstream bug in sourceware.org/bugzilla against glibc upstream and ask them to fix the en_CA locale. That way once it gets fixed in upstream it can trickle down into Fedora, and RHEL, and backports can be done.
Comment 8 James B. Byrne 2014-03-06 12:03:33 EST
> It's a bug if the locale doesn't meet the government standards. 
> File a bug against RHEL6 and it will go into the queue of bugs to get
> fixed. Please understand that the bug will be prioritized against other
> bugs to fix in the release.


This implies that I am to file another identical bug report on some other venue.  Where is that venue if not here?

The upstream maintainers have stated that the legal requirement of the Canadian Government has no bearing on what they, the maintainers, think suitable for the Canadian LOCALES en_CA and fr_CA.  See: https://sourceware.org/bugzilla/show_bug.cgi?id=12731#c4

So, how does this bug get fixed?
Comment 9 Carlos O'Donell 2014-03-06 14:26:41 EST
(In reply to James B. Byrne from comment #8)
> > It's a bug if the locale doesn't meet the government standards. 
> > File a bug against RHEL6 and it will go into the queue of bugs to get
> > fixed. Please understand that the bug will be prioritized against other
> > bugs to fix in the release.
> 
> This implies that I am to file another identical bug report on some other
> venue.  Where is that venue if not here?

This bug is an RFE, for fixing the documentation.

I'm requesting you file a second bug, again with Red Hat, about fixing the locale or adding a new locale. Since that additional locale is orthogonal to documenting this and making it easier.

Does that answer your question?

> The upstream maintainers have stated that the legal requirement of the
> Canadian Government has no bearing on what they, the maintainers, think
> suitable for the Canadian LOCALES en_CA and fr_CA.  See:
> https://sourceware.org/bugzilla/show_bug.cgi?id=12731#c4

One maintainer responding does not constitute consensus. Upstream is a consensus driven community. Also I am Canadian, and as a Canadian I expect %Y-%m-%d also.
 
> So, how does this bug get fixed?

I fix it upstream after gathering consensus.

Cheers,
Carlos.
Comment 24 Marko Myllynen 2016-11-07 05:52:52 EST
RHEL 7.3 now contains updated locale/localedef man pages which should provide information about locale installation, localedef, and locale source file format.

Thanks.

Note You need to log in before you can comment on or make changes to this bug.