Bug 1202705 - rsyslog mmopenshift does not log namespace in scaled gears
Summary: rsyslog mmopenshift does not log namespace in scaled gears
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers
Version: 2.2.0
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
: ---
Assignee: Timothy Williams
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-17 09:28 UTC by Josep 'Pep' Turro Mauri
Modified: 2019-07-11 08:47 UTC (History)
8 users (show)

Fixed In Version: rsyslog7-7.4.10-4.el6op
Doc Type: Bug Fix
Doc Text:
When a message came through rsyslog from a gear with a new UID, the mmopenshift plug-in set the metadata keys and values in cache for each environment variable specified in the template. If an environment variable was not present at this time, the key would not be created and stored in the metadata cache. This bug fix updates the mmopenshift plug-in so that on every message, if a key expected to be present in the metadata cache is not present, the plug-in checks for the environment variable. If the environment variable exists, the plug-in sets the metadata key to the value of the environment variable. Otherwise, it checks again on the next message. As a result, new gears from a scaled application may report a NULL namespace a few times until the OPENSHIFT_NAMESPACE environment variable is defined. Additionally, any other environment variables in the template will have the same checks. Note that all environment variables specified in the template are expected to be present for all applications. If an environment variable is specified that does not exist for an application, a slight performance degradation may be seen as the environment variable will be checked for on every message.
Clone Of:
Environment:
Last Closed: 2015-07-21 19:12:06 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1566723 0 None None None Never
Red Hat Product Errata RHBA-2015:1463 0 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 2.2.6 bug fix and enhancement update 2015-07-21 23:11:33 UTC

Description Josep 'Pep' Turro Mauri 2015-03-17 09:28:23 UTC
Description of problem:

mmopenshift fails to log namespace information for additional gears added after user varialbes exist in an application.

Version-Release number of selected component (if applicable):
rubygem-openshift-origin-controller-1.31.5.1-1.el6op.noarch
rsyslog7-mmopenshift-7.4.10-3.el6op.x86_64

How reproducible:
Always

Steps to Reproduce:
0.- configure mmopenshift with defaults as per the Admin Guide

1.- create a scalable app
    rhc app create -s myapp ruby-1.9

2.- add a custom env var to it
    rhc set-env MYVAR=something -a myapp

3.- add new gears to the app. Either by scaling it up:
    rhc cartridge scale -a myapp ruby-1.9 2

    or add a DB:
    rhc cartridge add -a myapp mysql-5.5

Actual results:
rsyslog messages coming from the new gears lack namespace information (ns is empty)

  2015-03-17T05:02:48.264634-04:00 vm ruby[22631]: app=myapp ns= appUuid=5507ed80e659c552f8000022 gearUuid=5507edaae659c552f8000045 [Tue Mar 17 05:02:48 2015] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations

  2015-03-17T05:07:59.752681-04:00 vm mysql[26778]: app=myapp ns= appUuid=5507ed80e659c552f8000022 gearUuid=5507eedde659c552f8000055 150317  5:07:59 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections.

Expected results:
New gears should log namespace details

Additional info:

When an application has user variables, Application#calculate_ops adds a PatchUserEnvVarsOp that pushes them to the new gear via rsync when OPENSHIFT_NAMESPACE is still not available in the new gear's env. This rsync triggers a log entry when the namespace information is not yet available inside the gear:

  Mar 17 05:07:49 vm rsyslogd-3000: mmopenshift: error opening file /var/lib/openshift/5507eedde659c552f8000055/.env/OPENSHIFT_NAMESPACE: No such file or directory

  2015-03-17T05:07:49.191439-04:00 vm oo-trap-user[26400]: app=myapp ns= appUuid=5507ed80e659c552f8000022 gearUuid=5507eedde659c552f8000055 oo_login: None from: 192.168.122.184 gearUuid=5507eedde659c552f8000055 cmd=rsync --server -prSe.iLs --delete . .env/user_vars

The available information for the gear is added to mmopenshift's gear info hash/cache (without namespace information) and all subsequent messages from that gear (uid) have an empty ns, until rsyslog is restarted (and information from gears has to be cached again).

Comment 2 Anping Li 2015-05-18 08:18:21 UTC
Verified and pass with the package in brew 
1) rpm package
[root@broker ~]# rpm -qa|grep syslog7
rsyslog7-mmopenshift-7.4.10-3.el6op.bz1202705.1.x86_64
rsyslog7-7.4.10-3.el6_6.x86_64
2) create scale application and scale up gear, and check the syslog message, the ns field are set correctly.

cat /var/log/openshift_gears

May 18 15:47:32 broker mysql[12388]: app=myapp ns=anlidom appUuid=555998714add7114b7000030 gearUuid=555999054add7114b7000065 150518 15:47:32 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections.
May 18 15:47:32 broker mysql[12388]: app=myapp ns=anlidom appUuid=555998714add7114b7000030 gearUuid=555999054add7114b7000065 150518 15:47:32 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections.

[root@broker ~]# cat /var/log/messages|grep "ns= "
[root@broker ~]#

Comment 6 errata-xmlrpc 2015-07-21 19:12:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1463.html


Note You need to log in before you can comment on or make changes to this bug.