Bug 1113479 - [RHS-LOGGING]: Redirecting logs to syslog floods meaningless fields in the log messages
Summary: [RHS-LOGGING]: Redirecting logs to syslog floods meaningless fields in the l...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: glusterfs
Version: rhgs-3.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Shyamsundar
QA Contact: storage-qa-internal@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-06-26 09:22 UTC by shylesh
Modified: 2018-02-06 02:41 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-02-06 02:41:06 UTC
Embargoed:


Attachments (Terms of Use)

Description shylesh 2014-06-26 09:22:43 UTC
Description of problem:
Setting logger=syslog makes all the gluster logs to be redirected to /var/log/messages in which a message contains some meaningless fields


Version-Release number of selected component (if applicable):

3.6.0.22-1.el6rhs


How reproducible:
Always

Steps to Reproduce:
1. start glusterd with --logger set to syslog 
   glusterd --logger=syslog  --log-format=with-msg-id --log-level=DEBUG
2. check the /var/log/messages 

Actual results:

Log message has few unknow fields:- 

1. @cee
2. "gf_code"
3. "9999"
4. gf_message"
5. "devel error"




Expected results:


Additional info:
55 (from 0)", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:1415:select_server_supported_programs] 0-rep-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[rpc-clnt.c:1759:rpc_clnt_reconfig] 0-rep-client-1: changing port to 49153 (from 0)", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:1200:client_setvolume_cbk] 0-rep-client-0: Connected to rep-client-0, attached to remote volume '/home/g5'.", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:1212:client_setvolume_cbk] 0-rep-client-0: Server and Client lk-version numbers are not same, reopening the fds", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:188:client_set_lk_version_cbk] 0-rep-client-0: Server lk version = 1", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:1415:select_server_supported_programs] 0-rep-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 home-g6[30264]: @cee: {"msg": "[client_t.c:184:gf_client_get] 0-rep-server: client_uid=rhs-client4.lab.eng.blr.redhat.com-30296-2014/06/26-09:07:15:202863-rep-client-1-0-0", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 home-g6[30264]: @cee: {"msg": "[server-handshake.c:578:server_setvolume] 0-rep-server: accepted client from rhs-client4.lab.eng.blr.redhat.com-30296-2014/06/26-09:07:15:202863-rep-client-1-0-0 (version: 3.6.0.22)", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:1200:client_setvolume_cbk] 0-rep-client-1: Connected to rep-client-1, attached to remote volume '/home/g6'.", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:1212:client_setvolume_cbk] 0-rep-client-1: Server and Client lk-version numbers are not same, reopening the fds", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 home-g6[30264]: @cee: {"msg": "[client_t.c:184:gf_client_get] 0-rep-server: client_uid=rhs-client4.lab.eng.blr.redhat.com-30296-2014/06/26-09:07:15:202863-rep-client-1-0-0", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[client-handshake.c:188:client_set_lk_version_cbk] 0-rep-client-1: Server lk version = 1", "gf_code": "9999", "gf_message": "devel error"}
Jun 26 14:37:19 rhs-client4 glustershd[30302]: @cee: {"msg": "[afr-self-heald.c:1735:afr_dir_exclusive_crawl] 0-rep-replicate-0: Another crawl is in progress for rep-client-1 while attempting INDEX heal on rep-client-1", "gf_code": "9999", "gf_message": "devel error"}

Comment 3 Shyamsundar 2014-07-01 13:23:16 UTC
Setting the logger to syslog was needed for a few reasons,

- Log consolidation to a central location
- Better log parsing using tools like lumber jack
- Better log representation using CEE format etc.

The older implementation of syslog forces the printing of all messages in the CEE format. This causes a confusion of sorts as the other pieces like lumber jack etc. is not in place.

The current implementation takes lumber jack out of the equation and hence requires that all messages be converted to the new format before syslog be used as the logger framework.

In other words, currently changing the logger to syslog is not supported till the all the logging changes to the gf_msg framework.

One possible way out of this is to change the older log framework to not print in CEE framework, so that this cacophony of messages is not seen, till all messages are put in the right format.


Note You need to log in before you can comment on or make changes to this bug.