1286633 – ausearch can't parse time in non-US locale and UTF-8 enconding

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1286633 - ausearch can't parse time in non-US locale and UTF-8 enconding

Summary: ausearch can't parse time in non-US locale and UTF-8 enconding

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	audit
Sub Component:
Version:	7.1
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Steve Grubb
QA Contact:	Ondrej Moriš
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-11-30 11:34 UTC by Karel Srot
Modified:	2016-11-04 06:12 UTC (History)
CC List:	4 users (show)
Fixed In Version:	audit-2.5.2-1.el7
Doc Type:	Bug Fix
Doc Text:
Clone Of:	1197235
Environment:
Last Closed:	2016-11-04 06:12:39 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
The list of locales for which ausearch does not parse time (12.04 KB, text/plain) 2016-03-24 13:45 UTC, Ondrej Moriš	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2016:2418	0	normal	SHIPPED_LIVE	audit bug fix and enhancement update	2016-11-03 13:58:32 UTC

Description Karel Srot 2015-11-30 11:34:56 UTC

Present also with audit-2.4.1-5.el7.x86_64

+++ This bug was initially created as a clone of Bug #1197235 +++

Description of problem:

# export LANG="ja_JP.UTF8"

# python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); import time; print time.strftime("%x %X", time.localtime(time.time()))'
2015年02月27日 22時38分08秒

# /sbin/ausearch -m avc -ts 2015年02月27日 22時38分08秒
Invalid start time (22時38分08秒). Hour, Minute, and Second are required.


Version-Release number of selected component (if applicable):
audit-2.3.7-5.el6.x86_64

How reproducible:
always

--- Additional comment from Steve Grubb on 2015-10-13 11:42:21 EDT ---

Increased buffer size in upstream commit 1069.

Comment 3 Ondrej Moriš 2016-03-24 13:29:00 UTC

As mentioned in BZ#1197235#c4 - BZ#1197235#c9, there are other locales for which ausearch cannot parse date and time values, current number with audit-2.4.5-2 is 141/735. The list of problematic locales with audit-2.4.5-2 is attached to this BZ, format of the file is as follows:

$LANG|$(date "+%x")|$(date "+%X")|$(date)

Comment 4 Ondrej Moriš 2016-03-24 13:45:41 UTC

Created attachment 1139973 [details]
The list of locales for which ausearch does not parse time

Comment 5 Steve Grubb 2016-03-29 14:38:55 UTC

I added:

  DL=`expr length "$D"`
  TL=`expr length "$T"`

to the test script and had it to output the lengths when failing. What I found was that the maximum time size was 54 and the maximum date size was 49. So, I think increasing the buffer to 64 bytes should cover this.

Comment 6 Steve Grubb 2016-03-29 15:09:21 UTC

The change to 64 bytes works for all but 21 locales. They have such a weird time format that I'm not sure we should do anything. Making it correct for these locales will be very complicated.

ff_SN|29/03/2016|10|10:52|5|maw mbo 29 10:52:28 EDT 2016|
ff_SN.utf8|29/03/2016|10|10:52|5|maw mbo 29 10:52:30 EDT 2016|
gd_GB|29/03/16|8|10:53:48 m|10|DiM M�rt 29 10:53:49 EDT 2016|
gd_GB.iso885915|29/03/16|8|10:53:49 m|10|DiM M�rt 29 10:53:50 EDT 2016|
gd_GB.utf8|29/03/16|8|10:53:50 m|10|DiM Màrt 29 10:53:51 EDT 2016|
hy_AM|03/29/16|8|10:55:08 |9|Երք Մար 29 10:55:09 EDT 2016|
hy_AM.armscii8|03/29/16|8|10:55:09 |9|��� س� 29 10:55:10 EDT 2016|
hy_AM.utf8|03/29/16|8|10:55:10 |9|Երք Մար 29 10:55:11 EDT 2016|
lzh_TW|廿十六年三月廿九日|27|十時57分36秒|16|公曆 20十六年 三月 廿九日 週二 十時57分37秒|
lzh_TW.utf8|廿十六年三月廿九日|27|十時57分37秒|16|公曆 20十六年 三月 廿九日 週二 十時57分38秒|
my_MM|၂၀၁၆ မတ် ၂၉ အင်္ဂါ|48|၁၀:၅၈:၂၇ နံနက်|36|ဂါ မတ် 29 10:58:28 EDT 2016|
my_MM.utf8|၂၀၁၆ မတ် ၂၉ အင်္ဂါ|48|၁၀:၅၈:၂၈ နံနက်|36|ဂါ မတ် 29 10:58:29 EDT 2016|
nb_NO.iso88591|29. mars 2016|13|kl. 10.58 -0400|15|ti. 29. mars 10:58:38 -0400 2016|
nb_NO.utf8|29. mars 2016|13|kl. 10.58 -0400|15|ti. 29. mars 10:58:39 -0400 2016|
nn_NO|29. mars 2016|13|kl. 10.59 -0400|15|ty. 29. mars 10:59:22 -0400 2016|
nn_NO.iso88591|29. mars 2016|13|kl. 10.59 -0400|15|ty. 29. mars 10:59:23 -0400 2016|
nn_NO.utf8|29. mars 2016|13|kl. 10.59 -0400|15|ty. 29. mars 10:59:25 -0400 2016|
no_NO|29. mars 2016|13|kl. 10.59 -0400|15|ti. 29. mars 10:59:27 -0400 2016|
norwegian|29. mars 2016|13|kl. 10.59 -0400|15|ti. 29. mars 10:59:29 -0400 2016|
sa_IN|मंगलवासर: 29 मार्च 2016|49|11:01:12  EDT|13|मंगल: मार्च 29 11:01:13 EDT 2016|
sa_IN.utf8|मंगलवासर: 29 मार्च 2016|49|11:01:13  EDT|13|मंगल: मार्च 29 11:01:14 EDT 2016|

Comment 7 Ondrej Moriš 2016-03-30 16:12:42 UTC

Using your test case (strftime() -> strptime()) I just reported BZ#1322292 for glibc as you suggested. Reducing number to 21 is great, actually 2 of them are most likely broken in glibc (my_MM and my_MM.utf8). From 795 locales 19 cannot be parsed correctly now, that is not a bad score. I would say we do not need to care about exotic ones (eg. Scottish Gaelic) but I am a bit worried about Norwegian ones.

Comment 8 Steve Grubb 2016-03-30 16:27:25 UTC

Yeah, but the issue with the Norwegian times is that it appears to print the gmt offset after the time. I honestly don't think anyone is that meticulous that they always write down the gmt offset every time they write down time. There are others that use EDT (in ascii no less) which again I think is unreasonable for a person to type.

Maybe we should not support locales and just require time as xx:yy[:zz.]

Comment 9 Paul Moore 2016-03-30 17:18:47 UTC

I might suggest looking at, and adopting, RFC 3339.

 * https://www.ietf.org/rfc/rfc3339.txt

Comment 10 Steve Grubb 2016-04-29 17:25:55 UTC

audit-2.5.2-1.el7 has been built to address this issue.

Comment 14 errata-xmlrpc 2016-11-04 06:12:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2418.html

Note You need to log in before you can comment on or make changes to this bug.