Bug 89292 - many UTF-8 symbols appear garbled in syslog
many UTF-8 symbols appear garbled in syslog
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: sysklogd (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Jason Vas Dias
Brian Brock
: i18n
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2003-04-22 06:27 EDT by Alexander Smolin
Modified: 2007-11-30 17:10 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2004-08-18 15:12:14 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
test program for syslog(3) (297 bytes, text/plain)
2003-04-22 06:31 EDT, Alexander Smolin
no flags Details
wrong content-type above (297 bytes, application/octet-stream)
2003-04-22 06:34 EDT, Alexander Smolin
no flags Details
Quick hacked patch (1010 bytes, patch)
2004-06-04 12:26 EDT, SEKINE Tatsuo
no flags Details | Diff
printline patch for sysklogd-1.4.1-18 (707 bytes, patch)
2004-06-14 12:58 EDT, SEKINE Tatsuo
no flags Details | Diff

  None (edit)
Description Alexander Smolin 2003-04-22 06:27:57 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225

Description of problem:
When I use ru_RU.UTF-8 locale localized messages apperar in system logs  mostly
unreadable. The same messsages look perfectly good on the console.
I have undertaken some quick investigation on this matter:
1. initscripts package uses initlog(8) program to send messages to syslog
2. initlog program uses SYSLOG(3) function from glibc
3. I have written a simple program to test this function in UTF-8 environment
(see attachment). The result is the same: UTF encoded strings get damaged
somewhere on thir way to syslog.
4. Have lost myself in glibc source code ... :((
Is that a glibc bug?

Version-Release number of selected component (if applicable):
glibc-2.3.2-27.9

How reproducible:
Always

Steps to Reproduce:
1.export LANG=ru_RU.UTF-8
2.service cups restart (or any other action that should produce a message to syslog)
3. tail -f /var/log/messages
    

Actual Results:  Some chars in the message are not UTF-8 encoded. Actually they
appear as three digit numbers between unreadable symbols

Expected Results:  UTF-8 encoded message in log

Additional info:

As a temporary workaround I had to add a couple of lines in /etc/init.d/functions:

export LC_MESSAGES=C
export LC_TIME=C

But unfortunately this solved only logging from initscripts (95%), but logging
from other applications are still unreadable (5%).
Comment 1 Alexander Smolin 2003-04-22 06:31:22 EDT
Created attachment 91223 [details]
test program for syslog(3)
Comment 2 Alexander Smolin 2003-04-22 06:34:15 EDT
Created attachment 91224 [details]
wrong content-type above
Comment 3 Jakub Jelinek 2003-04-22 06:54:06 EDT
This has nothing to do with glibc, which just creates the requrested message
(similarly to what sprintf does - your testcase should setlocale (LC_ALL, "")
first before feeding UTF-8 chars to syslog BTW) and writes it to the desired recipients
(which is sometimes console, sometimes apps stderr and usually syslog'd socket).
It is syslogd which decides which characters to print and which should not be printed.
Comment 4 Alexander Smolin 2003-04-22 08:39:26 EDT
Oh, thank you for pointing to syslogd! For myself, the problem was happily
solved by a quick hack to printline() function of syslogd.c,
so looking forward for updated sysklogd package from RedHat :)
Comment 5 SEKINE Tatsuo 2003-04-25 12:27:33 EDT
I think that this is the same ofbug #61296.
Comment 6 SEKINE Tatsuo 2004-06-04 12:26:43 EDT
Created attachment 100873 [details]
Quick hacked patch

Still 0x00-0x1F, 0x7F, 0xFE, 0xFF are escaped.
See Also the ENCODING section of utf-8 man page.
Comment 7 SEKINE Tatsuo 2004-06-04 12:43:52 EDT
Comment #5 is my misunderstanding.
I also think that printline() function of syslogd.c should be fixed.

BTW, Please add keyword "i18n" (I can't change keyword :-<).
http://www.redhat.com/archives/fedora-devel-list/2004-May/msg00813.html

Please fix platform. This problem doesn't depend on architecture
(I also can't change platform).
Comment 8 SEKINE Tatsuo 2004-06-14 12:58:44 EDT
Created attachment 101117 [details]
printline patch for sysklogd-1.4.1-18

This patch fix printline() problems on sysklogd-1.4.1-17/sysklogd-1.4.1-18.
Comment 9 SEKINE Tatsuo 2004-06-14 13:07:04 EDT
Comment on attachment 101117 [details]
printline patch for sysklogd-1.4.1-18

Please ignore "Only in ..." lines.
I forgot "make clean".
Comment 10 Florian La Roche 2004-06-14 18:04:58 EDT
Would the fix in #6 be enough to get everything logged or do we have
to go to that patch that only handels '\n', but outputs everything
else? No security concern about this either?

greetings,

Florian La Roche
Comment 11 SEKINE Tatsuo 2004-06-14 22:21:51 EDT
What does mean "security concern"?
I can't access both #71170 and #112519.

Because I think that syslogging should be "AS IS", I prefer the '\n' 
only patch. But what is the best depends on what "security concern" 
means.

Comment 12 Jason Vas Dias 2004-08-18 14:45:14 EDT
This is still happening in FC3/rawhide latest build -
changed LANG to "fr_FR:UTF-8" in /etc/sysconfig/i18n,
then did 'service syslogd restart' - messages appear
fine in French on the terminal, but are garbled in log.
I'm investigating and will fix this time!
Comment 13 Jason Vas Dias 2004-08-18 15:12:14 EDT
 Whoops ! Sorry, please ignore previous entry. The encoding is now
 '#if 0'd out in syslogd. It was xterm not displaying unicode 
 correctly! 
 
 
Comment 14 Jay Turner 2004-09-01 23:29:41 EDT
An errata has been issued which should help the problem 
described in this bug report. This report is therefore being 
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, 
please follow the link below. You may reopen this bug report 
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2004-335.html

Note You need to log in before you can comment on or make changes to this bug.