Bug 1097748

Summary: Unable to differentiate syslog debug and trace logs from openshift-origin-node
Product: OpenShift Container Platform Reporter: Brenton Leanhardt <bleanhar>
Component: ContainersAssignee: Brenton Leanhardt <bleanhar>
Status: CLOSED ERRATA QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: high    
Version: 2.1.0CC: adellape, anli, dmace, jialiu, jokerman, libra-onpremise-devel, mmccomas
Target Milestone: ---Keywords: Upstream, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rubygem-openshift-origin-node-1.23.9.15-1.el6op Doc Type: Bug Fix
Doc Text:
Previously, it was difficult to distinguish between debug and trace log output on nodes because both log types were written using the local0 facility. This bug fix adds new settings to allow for better distinction between log types. Logs are now written using the facility defined by the PLATFORM_SYSLOG_FACILITY setting (default: LOG_LOCAL0), and, if trace logging is enabled, trace logs are now written using the facility defined by the PLATFORM_SYSLOG_TRACE_FACILITY setting (default: LOG_LOCAL1). Administrators can change these settings from their defaults by adding and defining them in the /etc/openshift/node.conf file.
Story Points: ---
Clone Of: 1097745 Environment:
Last Closed: 2014-08-26 13:52:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1097745    
Bug Blocks:    

Description Brenton Leanhardt 2014-05-14 12:50:33 UTC
+++ This bug was initially created as a clone of Bug #1097745 +++

Description of problem:

The SyslogLogger implementation in openshift-origin-node doesn't give enough context to allow rsyslog or other downstream systems to distinguish debug and trace logs: they both log to local0 using the debug priority.

I recommend logging trace messages using the local1 facility with the debug priority.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:

1. Configure a node to use Syslog logging by modifying /etc/openshift/node.conf:

PLATFORM_LOG_CLASS=SyslogLogger
PLATFORM_SYSLOG_TRACE_ENABLED=1
PLATFORM_SYSLOG_THRESHOLD=LOG_DEBUG

2. service ruby193-mcollective restart

3. Watch /var/log/messages

Actual results:

Debug and trace logs are combined.

Expected results:

The ability to split the debug/trace logs by modifying /etc/rsyslogd.conf, e.g.:

local1.*   /var/log/platform-trace.log

Additional info:

Comment 1 Brenton Leanhardt 2014-08-01 13:35:27 UTC
Upstream commit:

commit e8b4412f410dac386a2b31617d6de044842a34c5
Author: Dan Mace <ironcladlou>
Date:   Wed May 14 08:37:28 2014 -0400

    Use the local1 facility for Syslog trace logs
    
    Allow trace log segregation by logging trace messages using the local1
    facility.
    
    Resolve bug: https://bugzilla.redhat.com/show_bug.cgi?id=1097745

Comment 4 Anping Li 2014-08-04 06:37:44 UTC
Verified and pass on puddle-2-1-2014-08-01

1) Add Sys Log Configure values to node.conf 
PLATFORM_LOG_CLASS=SyslogLogger
PLATFORM_SYSLOG_TRACE_ENABLED=1
PLATFORM_SYSLOG_THRESHOLD=LOG_DEBUG
2) add local1.* /var/log/platform-trace.log in /etc/rsyslog.conf

3) 
A: some logs are recorded in /var/log/message

Aug  4 02:25:07 node1 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Aug  4 02:25:07 node1 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="17620" x-info="http://www.rsyslog.com"] start
Aug  4 02:25:17 node1 dhclient[977]: DHCPREQUEST on eth0 to 192.168.55.3 port 67 (xid=0x25e7f84b)
Aug  4 02:25:17 node1 dhclient[977]: DHCPACK from 192.168.55.3 (xid=0x25e7f84b)
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, mysql, 5.5, 0.2.13.1) from /usr/libexec/openshift/cartridges/mysql
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, perl, 5.10, 0.0.15) from /usr/libexec/openshift/cartridges/perl
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, jenkins, 1, 0.0.11) from /usr/libexec/openshift/cartridges/jenkins
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, python, 2.6, 0.0.17) from /usr/libexec/openshift/cartridges/python
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, haproxy, 1.4, 0.0.17.2) from /usr/libexec/openshift/cartridges/haproxy
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, jbosseap, 6, 0.0.14) from /usr/libexec/openshift/cartridges/jbosseap
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, mongodb, 2.4, 0.2.11) from /usr/libexec/openshift/cartridges/mongodb
Aug  4 02:25:18 node1 openshift-platform[17694]: openshift-agent: Installed cartridge (redhat, php, 5.4, 0.0.16.1) from /usr/libexec/openshift/cartridges/php
@  

B: some logs are recorded in /var/log/platform-trace.log 

[root@node1 node]# tailf /var/log/platform-trace.log 
Aug  4 02:25:18 node1 openshift-platform[17694]: oo_spawn running shopt -s dotglob; /bin/cp --recursive --no-dereference --preserve=mode,ownership,timestamps,links /usr/libexec/openshift/cartridges/jbosseap/* /var/lib/openshift/.cartridge_repository/redhat-jbosseap/0.0.14: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 13>, :err=>#<IO:fd 11>}
Aug  4 02:25:18 node1 openshift-platform[17694]: oo_spawn running shopt -s dotglob; /bin/cp --recursive --no-dereference --preserve=mode,ownership,timestamps,links /usr/libexec/openshift/cartridges/mongodb/* /var/lib/openshift/.cartridge_repository/redhat-mongodb/0.2.11: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 13>, :err=>#<IO:fd 11>}
Aug  4 02:25:18 node1 openshift-platform[17694]: oo_spawn running shopt -s dotglob; /bin/cp --recursive --no-dereference --preserve=mode,ownership,timestamps,links /usr/libexec/openshift/cartridges/php/* /var/lib/openshift/.cartridge_repository/redhat-php/0.0.16.1: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 13>, :err=>#<IO:fd 11>}

Comment 6 errata-xmlrpc 2014-08-26 13:52:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1095.html