Bug 708095 - the default date string fails the date picker's valid date string test for LANG=or_IN.utf8
Summary: the default date string fails the date picker's valid date string test for LA...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: subscription-manager
Version: 5.7
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Chris Duryee
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 675214 710615
TreeView+ depends on / blocked
 
Reported: 2011-05-26 18:00 UTC by John Sefler
Modified: 2011-07-21 12:30 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 710615 (view as bug list)
Environment:
Last Closed: 2011-07-21 08:46:06 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
The default date string fails the format error for LANG=or_IN (92.49 KB, image/png)
2011-05-26 18:00 UTC, John Sefler
no flags Details
This screenhot seems to indicate Oriya (or_IN) is available in RHEL57 (170.64 KB, image/png)
2011-05-26 19:39 UTC, John Sefler
no flags Details
screen shot with a working date string format in the date picker (74.12 KB, image/png)
2011-06-03 21:30 UTC, John Sefler
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:1078 0 normal SHIPPED_LIVE new package: subscription-manager 2011-07-21 08:45:07 UTC

Description John Sefler 2011-05-26 18:00:35 UTC
Created attachment 501154 [details]
The default date string fails the format error for LANG=or_IN

Description of problem:
The default date string for some languages triggers the "Invalid date format" error dialog. 


Version-Release number of selected component (if applicable):
I found this bug against a rhel6 install with sm built from the master branch.  However I am reporting it against rhel57 because I suspect the problem exist there too.

How reproducible:


Steps to Reproduce:
1. register to any candlepin server
2. I then used the following loop to start subscription-manager-gui in each language
# for l in en_US de_DE es_ES fr_FR it_IT ja_JP ko_KR pt_BR ru_RU zh_CN zh_TW as_IN bn_IN hi_IN mr_IN gu_IN kn_IN ml_IN or_IN pa_IN ta_IN te_IN; do echo ""; echo ""; echo "# LANG=$l.UTF8 subscription-manager-gui"; LANG=$l.UTF8 subscription-manager-gui; done;

3. click to the All Available Subscriptions tab
4. click the date/calendar picker WITH THE DEFAULT DATE STRING IN THE TEXT FIELD.

  
Actual results:
For one language (or_IN), the Invalid date format message shows up with an example of todays date which should be valid. It is not valid.

Expected results:
For all languages, the calendar widget should open to today's date

Additional info:
The severe consequence of this bug is that the user is now blocked from searching for subscriptions in LANG=or_IN

Comment 1 John Sefler 2011-05-26 18:13:35 UTC
found with version...
[root@jsefler-onprem-6server ~]# rpm -qa | grep subscription-manager
subscription-manager-gnome-0.96.1-1.git.198.99d1857.el6.x86_64
subscription-manager-firstboot-0.96.1-1.git.198.99d1857.el6.x86_64
subscription-manager-0.96.1-1.git.198.99d1857.el6.x86_64

Comment 2 Chris Duryee 2011-05-26 18:36:59 UTC
Is or_IN supported in 5.7? When I test this in 5.7, I get an "unknown locale" error.

Comment 3 Adrian Likins 2011-05-26 18:46:28 UTC
I see the same as comment #2 on 5.7 and/or f14

Comment 4 John Sefler 2011-05-26 19:39:52 UTC
Created attachment 501175 [details]
This screenhot seems to indicate Oriya (or_IN) is available in RHEL57

Comment 5 Chris Duryee 2011-05-27 13:55:35 UTC
Are you able to reproduce this in 5? If or_IN is supported, the bug may lie elsewhere. I'm not able to run anything with LANG=or_IN.

Comment 6 John Sefler 2011-05-27 14:29:24 UTC
LANG=or_IN.UTF8 subscription-manager-gui
           ^^^^ don't forget this

Here is a cut and paste of today's (6/27/2011) default date string for or_IN.UTF8
୨୭-୫-୧୧

Comment 7 Adrian Likins 2011-05-27 15:33:06 UTC
LANG=or_IN.UTF8 python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); import time; import datetime; print locale.getdefaultlocale(); print datetime.datetime.now().strftime("%x"); print time.strptime(datetime.datetime.now().strftime("%x"), "%x")'

('or_IN', 'UTF8')
୨୭-୫-୧୧
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python2.7/_strptime.py", line 454, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib64/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data '\xe0\xad\xa8\xe0\xad\xad-\xe0\xad\xab-\xe0\xad\xa7\xe0\xad\xa7' does not match format '%x'


So I'm not real sure what to do about that?

Comment 9 Dave Malcolm 2011-05-31 21:35:47 UTC
I can reproduce this on both RHEL5 with:
  python-2.4.3-42.el5
with output:
  ('or_IN', 'UTF8')
  ୩୧-୫-୧୧
  Traceback (most recent call last):
    File "<string>", line 1, in ?
    File "/usr/lib/python2.4/_strptime.py", line 293, in strptime
      raise ValueError("time data did not match format:  data=%s  fmt=%s" %
  ValueError: time data did not match format:  data=୩୧-୫-୧୧  fmt=%x

and on Fedora 15 with:
  python-2.7.1-5.fc15.x86_64
with:
  ('or_IN', 'UTF8')
  ୩୧-୫-୧୧
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/usr/lib64/python2.7/_strptime.py", line 454, in _strptime_time
      return _strptime(data_string, format)[0]
    File "/usr/lib64/python2.7/_strptime.py", line 325, in _strptime
      (data_string, format))
  ValueError: time data '\xe0\xad\xa9\xe0\xad\xa7-\xe0\xad\xab-\xe0\xad\xa7\xe0\xad\xa7' does not match format '%x'
  
The exact error message varies somewhat between the two, but in both cases, _strptime is failing.

The datetime.strftime method is implemented in Modules/datetimemodule.c, and wrapping time.strftime (with some special-case preprocessing of %z and %Z first)

"time.strftime" is implemented in Modules/timemodule.c by wrapping strftime from libc, returning a PyStringObject from the result.

"man strftime" lists:
       %x     The preferred date representation for the current locale without
              the time.

The "time" module (in timemodule.so) implements "time.strptime" (Modules/timemodule.c) by using _strptime.strptime, and it's this that's failing.

_strptime.TimeRE is a dictionary, mapping from format directives to regexes. In this case, 'x' is mapped thusly:
  'x': '\xe0\xad\xa7\xe0\xad\xad-\xe0\xad\xa9-\xe0\xad\xaf\xe0\xad\xaf',
The right-hand side value above doesn't look like a meaningful regex; it comes from:
  TimeRE.__init__:
     self.pattern(self.locale_time.LC_date)

where self.locale_time is a LocaleTime instance; its LC_date attribute is initialized as:
    self.LC_date = date_time[1]
in __calc_date_time using
  time.strftime("%x", time_tuple).lower()
This function puts a "magic" date through strftime multiple times, substituting values back with format values, locating the format values.

Copying _strptime.py to the pwd, and editing to add "print" statements, and rerunning the reproducer, it looks that LocaleTime isn't working at all in this locale:

For example, in "or_IN.UTF8"
  date_time: ['\xe0\xad\xa7\xe0\xad\xad %B \xe0\xad\xaf\xe0\xad\xaf \xe0\xad\xa7\xe0\xad\xa6:\xe0\xad\xaa\xe0\xad\xaa:\xe0\xad\xab\xe0\xad\xab %p %Z', '\xe0\xad\xa7\xe0\xad\xad-\xe0\xad\xa9-\xe0\xad\xaf\xe0\xad\xaf', '\xe0\xad\xa7\xe0\xad\\xa6:\xe0\xad\xaa\xe0\xad\xaa:\xe0\xad\xab\xe0\xad\xab %p']

as compared to, say, "en_US.UTF8":
  date_time: ['%a %d %b %Y %I:%M:%S %p %Z', '%m/%d/%Y', '%I:%M:%S %p']

It looks like glibc is formatting the dates via strftime("%x") in a way that doesn't embed arabic numerals, and so LocaleTime's attempts to figure these out fails.

Comment 12 Dave Malcolm 2011-05-31 22:18:44 UTC
Specifically, the Oriya digits are these Unicode code points:
  U+0B66 ORIYA DIGIT ZERO  (UTF-8: 0xE0 0xAD 0xA6)
  U+0B67 ORIYA DIGIT ONE   (UTF-8: 0xE0 0xAD 0xA7)                                                                                                                                                                                              
   ...
  U+0B6F ORIYA DIGIT NINE  (UTF-8: 0xE0 0xAD 0xAF)

So
  '\xe0\xad\xa9\xe0\xad\xa7-\xe0\xad\xab-\xe0\xad\xa7\xe0\xad\xa7'
is indeed today's date:
  '31-5-11'
expressed in Oriya digits

Comment 17 Chris Duryee 2011-06-03 15:29:51 UTC
Fixed on RHEL5.7 branch in 4445aa473cbe1782dd5b0cdabba950a4fc4723c4, version

Comment 18 Chris Duryee 2011-06-03 15:31:11 UTC
version 0.95.5.21:) my script snafu'd

also fixed in master, d85bf6e8a42

Comment 19 John Sefler 2011-06-03 21:30:01 UTC
Created attachment 502913 [details]
screen shot with a working date string format in the date picker

Verifying Version...
# rpm -qa | grep subscription-manager
subscription-manager-gnome-0.95.5.20-1.git.5.4445aa4.el5
subscription-manager-0.95.5.20-1.git.5.4445aa4.el5
subscription-manager-firstboot-0.95.5.20-1.git.5.4445aa4.el5


# yum install fonts-oriya.noarch

# LANG=or_IN.UTF8 subscription-manager-gui &

Notice in the screenshot that the dd/mm/yy format is being used.
Moreover the date picker works and the search/update/subscribe now works in the oriya gui.

Before I mark this bug verified, I need to know from Chris if the dd/mm/yy format seen in the screenshot is expected since Chris said in a comment 16 above that he was going to use DD-MM-YYYY.  I also wonder about the untranslated calendar month and the english date string in the lower left showing the next update time.

Comment 20 Chris Duryee 2011-06-06 12:51:11 UTC
DD/MM/YYYY is what en_GB is, I mistakenly thought it was DD-MM-YYYY in comment 16.

The workaround was done by modifying LC_TIME, which would affect all dates. This was intentional, in order to have the workaround be consistent in the UI.

Comment 21 John Sefler 2011-06-06 14:09:32 UTC
moving to VERIFIED

Comment 22 errata-xmlrpc 2011-07-21 08:46:06 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-1078.html

Comment 23 errata-xmlrpc 2011-07-21 12:30:47 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-1078.html


Note You need to log in before you can comment on or make changes to this bug.