Red Hat Bugzilla – Bug 89292
many UTF-8 symbols appear garbled in syslog
Last modified: 2007-11-30 17:10:31 EST
From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 Description of problem: When I use ru_RU.UTF-8 locale localized messages apperar in system logs mostly unreadable. The same messsages look perfectly good on the console. I have undertaken some quick investigation on this matter: 1. initscripts package uses initlog(8) program to send messages to syslog 2. initlog program uses SYSLOG(3) function from glibc 3. I have written a simple program to test this function in UTF-8 environment (see attachment). The result is the same: UTF encoded strings get damaged somewhere on thir way to syslog. 4. Have lost myself in glibc source code ... :(( Is that a glibc bug? Version-Release number of selected component (if applicable): glibc-2.3.2-27.9 How reproducible: Always Steps to Reproduce: 1.export LANG=ru_RU.UTF-8 2.service cups restart (or any other action that should produce a message to syslog) 3. tail -f /var/log/messages Actual Results: Some chars in the message are not UTF-8 encoded. Actually they appear as three digit numbers between unreadable symbols Expected Results: UTF-8 encoded message in log Additional info: As a temporary workaround I had to add a couple of lines in /etc/init.d/functions: export LC_MESSAGES=C export LC_TIME=C But unfortunately this solved only logging from initscripts (95%), but logging from other applications are still unreadable (5%).
Created attachment 91223 [details] test program for syslog(3)
Created attachment 91224 [details] wrong content-type above
This has nothing to do with glibc, which just creates the requrested message (similarly to what sprintf does - your testcase should setlocale (LC_ALL, "") first before feeding UTF-8 chars to syslog BTW) and writes it to the desired recipients (which is sometimes console, sometimes apps stderr and usually syslog'd socket). It is syslogd which decides which characters to print and which should not be printed.
Oh, thank you for pointing to syslogd! For myself, the problem was happily solved by a quick hack to printline() function of syslogd.c, so looking forward for updated sysklogd package from RedHat :)
I think that this is the same ofbug #61296.
Created attachment 100873 [details] Quick hacked patch Still 0x00-0x1F, 0x7F, 0xFE, 0xFF are escaped. See Also the ENCODING section of utf-8 man page.
Comment #5 is my misunderstanding. I also think that printline() function of syslogd.c should be fixed. BTW, Please add keyword "i18n" (I can't change keyword :-<). http://www.redhat.com/archives/fedora-devel-list/2004-May/msg00813.html Please fix platform. This problem doesn't depend on architecture (I also can't change platform).
Created attachment 101117 [details] printline patch for sysklogd-1.4.1-18 This patch fix printline() problems on sysklogd-1.4.1-17/sysklogd-1.4.1-18.
Comment on attachment 101117 [details] printline patch for sysklogd-1.4.1-18 Please ignore "Only in ..." lines. I forgot "make clean".
Would the fix in #6 be enough to get everything logged or do we have to go to that patch that only handels '\n', but outputs everything else? No security concern about this either? greetings, Florian La Roche
What does mean "security concern"? I can't access both #71170 and #112519. Because I think that syslogging should be "AS IS", I prefer the '\n' only patch. But what is the best depends on what "security concern" means.
This is still happening in FC3/rawhide latest build - changed LANG to "fr_FR:UTF-8" in /etc/sysconfig/i18n, then did 'service syslogd restart' - messages appear fine in French on the terminal, but are garbled in log. I'm investigating and will fix this time!
Whoops ! Sorry, please ignore previous entry. The encoding is now '#if 0'd out in syslogd. It was xterm not displaying unicode correctly!
An errata has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2004-335.html