Bug 1146237

Summary: Backport mmutf8fix module?
Product: Red Hat Enterprise Linux 7 Reporter: Chris Adams <linux>
Component: rsyslogAssignee: Radovan Sroka <rsroka>
Status: CLOSED ERRATA QA Contact: Stefan Dordevic <sdordevi>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: nparmar, pvrabec, rmeggins, rsroka, sdordevi
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 05:34:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1296594, 1313485    
Attachments:
Description Flags
patch backport none

Description Chris Adams 2014-09-24 20:19:28 UTC 
With the RHEL 7 version of rsyslog, you include the omelasticsearch module, which is great!  However, unfortunately syslog messages are not always proper UTF-8 (some latin1, some just garbage), and elasticsearch only accepts UTF-8.  Upstream rsyslog has added a "hack" module that strips out non-ASCII, non-UTF-8 characters and replaces them, and that makes the omelasticsearch module much more useful.

I took the mmutf8fix module from upstream version 7.6.4 (along with the necessary Makefile* and configure* changes), and it appears to work just fine with the RHEL 7 rsyslog-7.4.7 package (I actually patched the SRPM to build a new subpackage).  The rsyslog internal interfaces haven't changed between those versions.

Would it be possible for the RHEL package to have the mmutf8fix module added as well?
Comment 2 Tomas Heinrich 2014-09-25 12:44:12 UTC 
(In reply to Chris Adams from comment #0)
> Would it be possible for the RHEL package to have the mmutf8fix module added
> as well?

I *think* it would be possible, but can't say when it would happen.
Comment 4 Rich Megginson 2015-11-20 17:37:05 UTC 
(In reply to Chris Adams from comment #0)
> Would it be possible for the RHEL package to have the mmutf8fix module added
> as well?

+1
Comment 5 Rich Megginson 2016-02-03 14:54:45 UTC 
Please consider this bug for 7.3
Comment 9 Radovan Sroka 2016-02-29 17:04:13 UTC 
Created attachment 1131628 [details]
patch backport

Here is the patch.

It can be tested e.g. :

----------------------------------------------------------------

module( load="imuxsock")
input(type="imuxsock" socket="/tmp/socket" createpath="on")


module(load="mmutf8fix")
action(type="omfile" file="/tmp/test_orig.log")
action(type="mmutf8fix")
action(type="omfile" file="/tmp/test_fixed.log")

----------------------------------------------------------------

logger -u /tmp/socket "this is non utf8 character  >>>`echo -e "\xFF"`<<<"

and result will be:

/tmp/test_orig.log:
ibm-x3650m4-01-vm-02 root: this is non utf8 character  >>>ΓΏ<<<

/tmp/test_fixed.log:
ibm-x3650m4-01-vm-02 root: this is non utf8 character  >>> <<<
Comment 14 errata-xmlrpc 2016-11-04 05:34:59 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2401.html