Bug 89292 - many UTF-8 symbols appear garbled in syslog
Summary: many UTF-8 symbols appear garbled in syslog
Alias: None
Product: Fedora
Classification: Fedora
Component: sysklogd   
(Show other bugs)
Version: rawhide
Hardware: All Linux
Target Milestone: ---
Assignee: Jason Vas Dias
QA Contact: Brian Brock
Keywords: i18n
Depends On:
TreeView+ depends on / blocked
Reported: 2003-04-22 10:27 UTC by Alexander Smolin
Modified: 2007-11-30 22:10 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2004-08-18 19:12:14 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
test program for syslog(3) (297 bytes, text/plain)
2003-04-22 10:31 UTC, Alexander Smolin
no flags Details
wrong content-type above (297 bytes, application/octet-stream)
2003-04-22 10:34 UTC, Alexander Smolin
no flags Details
Quick hacked patch (1010 bytes, patch)
2004-06-04 16:26 UTC, SEKINE Tatsuo
no flags Details | Diff
printline patch for sysklogd-1.4.1-18 (707 bytes, patch)
2004-06-14 16:58 UTC, SEKINE Tatsuo
no flags Details | Diff

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2004:335 normal SHIPPED_LIVE Updated sysklogd package 2004-09-01 04:00:00 UTC

Description Alexander Smolin 2003-04-22 10:27:57 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225

Description of problem:
When I use ru_RU.UTF-8 locale localized messages apperar in system logs  mostly
unreadable. The same messsages look perfectly good on the console.
I have undertaken some quick investigation on this matter:
1. initscripts package uses initlog(8) program to send messages to syslog
2. initlog program uses SYSLOG(3) function from glibc
3. I have written a simple program to test this function in UTF-8 environment
(see attachment). The result is the same: UTF encoded strings get damaged
somewhere on thir way to syslog.
4. Have lost myself in glibc source code ... :((
Is that a glibc bug?

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.export LANG=ru_RU.UTF-8
2.service cups restart (or any other action that should produce a message to syslog)
3. tail -f /var/log/messages

Actual Results:  Some chars in the message are not UTF-8 encoded. Actually they
appear as three digit numbers between unreadable symbols

Expected Results:  UTF-8 encoded message in log

Additional info:

As a temporary workaround I had to add a couple of lines in /etc/init.d/functions:

export LC_TIME=C

But unfortunately this solved only logging from initscripts (95%), but logging
from other applications are still unreadable (5%).

Comment 1 Alexander Smolin 2003-04-22 10:31:22 UTC
Created attachment 91223 [details]
test program for syslog(3)

Comment 2 Alexander Smolin 2003-04-22 10:34:15 UTC
Created attachment 91224 [details]
wrong content-type above

Comment 3 Jakub Jelinek 2003-04-22 10:54:06 UTC
This has nothing to do with glibc, which just creates the requrested message
(similarly to what sprintf does - your testcase should setlocale (LC_ALL, "")
first before feeding UTF-8 chars to syslog BTW) and writes it to the desired recipients
(which is sometimes console, sometimes apps stderr and usually syslog'd socket).
It is syslogd which decides which characters to print and which should not be printed.

Comment 4 Alexander Smolin 2003-04-22 12:39:26 UTC
Oh, thank you for pointing to syslogd! For myself, the problem was happily
solved by a quick hack to printline() function of syslogd.c,
so looking forward for updated sysklogd package from RedHat :)

Comment 5 SEKINE Tatsuo 2003-04-25 16:27:33 UTC
I think that this is the same ofbug #61296.

Comment 6 SEKINE Tatsuo 2004-06-04 16:26:43 UTC
Created attachment 100873 [details]
Quick hacked patch

Still 0x00-0x1F, 0x7F, 0xFE, 0xFF are escaped.
See Also the ENCODING section of utf-8 man page.

Comment 7 SEKINE Tatsuo 2004-06-04 16:43:52 UTC
Comment #5 is my misunderstanding.
I also think that printline() function of syslogd.c should be fixed.

BTW, Please add keyword "i18n" (I can't change keyword :-<).

Please fix platform. This problem doesn't depend on architecture
(I also can't change platform).

Comment 8 SEKINE Tatsuo 2004-06-14 16:58:44 UTC
Created attachment 101117 [details]
printline patch for sysklogd-1.4.1-18

This patch fix printline() problems on sysklogd-1.4.1-17/sysklogd-1.4.1-18.

Comment 9 SEKINE Tatsuo 2004-06-14 17:07:04 UTC
Comment on attachment 101117 [details]
printline patch for sysklogd-1.4.1-18

Please ignore "Only in ..." lines.
I forgot "make clean".

Comment 10 Florian La Roche 2004-06-14 22:04:58 UTC
Would the fix in #6 be enough to get everything logged or do we have
to go to that patch that only handels '\n', but outputs everything
else? No security concern about this either?


Florian La Roche

Comment 11 SEKINE Tatsuo 2004-06-15 02:21:51 UTC
What does mean "security concern"?
I can't access both #71170 and #112519.

Because I think that syslogging should be "AS IS", I prefer the '\n' 
only patch. But what is the best depends on what "security concern" 

Comment 12 Jason Vas Dias 2004-08-18 18:45:14 UTC
This is still happening in FC3/rawhide latest build -
changed LANG to "fr_FR:UTF-8" in /etc/sysconfig/i18n,
then did 'service syslogd restart' - messages appear
fine in French on the terminal, but are garbled in log.
I'm investigating and will fix this time!

Comment 13 Jason Vas Dias 2004-08-18 19:12:14 UTC
 Whoops ! Sorry, please ignore previous entry. The encoding is now
 '#if 0'd out in syslogd. It was xterm not displaying unicode 

Comment 14 Jay Turner 2004-09-02 03:29:41 UTC
An errata has been issued which should help the problem 
described in this bug report. This report is therefore being 
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, 
please follow the link below. You may reopen this bug report 
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.