Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1096900

Summary:	Log messages are being interpolated by syslog_logger.rb, causing random app operation failures
Product:	OpenShift Container Platform	Reporter:	Brenton Leanhardt <bleanhar>
Component:	Containers	Assignee:	Luke Meyer <lmeyer>
Status:	CLOSED ERRATA	QA Contact:	libra bugs <libra-bugs>
Severity:	medium	Docs Contact:
Priority:	high
Version:	2.1.0	CC:	bmeng, dakini, dmace, jokerman, libra-onpremise-devel, pruan, xiama
Target Milestone:	---	Keywords:	Upstream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	rubygem-openshift-origin-node-1.23.9.9-1.el6op	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1095914	Environment:
Last Closed:	2014-06-18 20:00:15 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1095914
Bug Blocks:

Description Brenton Leanhardt 2014-05-12 15:47:24 UTC

+++ This bug was initially created as a clone of Bug #1095914 +++

Description of problem:

The syslog_logger.rb implementation in openshift-origin-node is making incorrect use of the Ruby syslog library. The code is treating the format string parameter as a formatted message parameter. This causes the raw input string (a log message) to be interpolated underneath in the syslog implementation. Any log messages dispatched via this code which contain the '%' character can result in exceptions unless followed by another character that happens to make the string conform to the printf substitution rules. These exceptions can occur anywhere in the openshift-origin-node code, or the MCollective agent.

The code should be changed to use a generic '%s' format string, passing the processed log message as the format args. This will make the call log the input verbatim.

Version-Release number of selected component (if applicable):


How reproducible:

Configure platform logging for syslog via /etc/openshift/node.conf:

PLATFORM_LOG_CLASS=SyslogLogger
PLATFORM_SYSLOG_THRESHOLD=LOG_INFO
PLATFORM_SYSLOG_TRACE_ENABLED=1
PLATFORM_LOG_FILE=/var/log/openshift/node/platform.log
PLATFORM_LOG_LEVEL=INFO
PLATFORM_TRACE_LOG_FILE=/var/log/openshift/node/platform-trace.log
PLATFORM_TRACE_LOG_LEVEL=DEBUG
PLATFORM_LOG_CONTEXT_ENABLED=1
PLATFORM_LOG_CONTEXT_ATTRS=request_id,app_uuid

One example that happens to easily produce the offending log messages is:

rhc app create sinkerror jbossas-7 -pp --no-git --noprompt --from-code=git://github.com/openshift/kitchensink-example.git

Steps to Reproduce:
1.
2.
3.

Actual results:

The MCollective agent will crash with an exception such as:

May  8 12:45:45 ex-std-node3 mcollectived[1965]: agents.rb:139:in `rescue in block in dispatch' /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.23.9/lib/openshift-origin-node/utils/logger/syslog_logger.rb:61:in `log'
...

The app will fail to be created due to the crash.

Expected results:

Correct log output to Syslog and a successfully created application.

Additional info:

--- Additional comment from Dan Mace on 2014-05-08 16:24:24 EDT ---

https://github.com/openshift/origin-server/pull/5401

--- Additional comment from Meng Bo on 2014-05-08 23:36:26 EDT ---

Checked on devenv-stage_830, issue has been fixed.

With the platform log send to syslog set as comment#0 in node.conf, created the kitchensink jbossas app several times.

And no error found in /var/log/ruby193-mcollective.log

Move bug to verified.

Comment 1 Luke Meyer 2014-05-23 13:45:31 UTC

https://github.com/openshift/enterprise-server/pull/283

Comment 2 Luke Meyer 2014-06-12 17:16:19 UTC

commit 56a50d0fc7c1a98af1f5a87fb5d2def418a55274
Commit:     Luke Meyer <lmeyer>
CommitDate: Thu May 29 11:59:50 2014 -0400

    syslog_logger: Prevent unwanted syslog message interpolation #cherrypick

    origin-server:
    commit f2328073c329defdb91e2aa009994bdf84570cbe
    Author: Dan Mace <ironcladlou>
    Date:   Thu May 8 16:15:49 2014 -0400

    Prevent unwanted syslog message interpolation

    Stop treating raw log input as a syslog format string, and instead
    pass the input as a format arg to the '%s' format string. This will
    prevent syslog from trying to interpolate random log messages as
    format strings.

    Resolve bug https://bugzilla.redhat.com/show_bug.cgi?id=1095914

Comment 3 Ma xiaoqiang 2014-06-13 07:52:32 UTC

Check on puddle [2.1.z/2014-06.12]

With the platform log send to syslog set as comment#0 in node.conf, create eapmongo app with jbosseap-6 and mongodb-2.4 cartridges ten times. 

No error found in /var/log/messages 

Move bug to verified.

Comment 5 errata-xmlrpc 2014-06-18 20:00:15 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2014-0764.html