Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1202705 - rsyslog mmopenshift does not log namespace in scaled gears
rsyslog mmopenshift does not log namespace in scaled gears
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers (Show other bugs)
2.2.0
All Linux
high Severity medium
: ---
: ---
Assigned To: Timothy Williams
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-03-17 05:28 EDT by Josep 'Pep' Turro Mauri
Modified: 2015-08-07 10:31 EDT (History)
8 users (show)

See Also:
Fixed In Version: rsyslog7-7.4.10-4.el6op
Doc Type: Bug Fix
Doc Text:
When a message came through rsyslog from a gear with a new UID, the mmopenshift plug-in set the metadata keys and values in cache for each environment variable specified in the template. If an environment variable was not present at this time, the key would not be created and stored in the metadata cache. This bug fix updates the mmopenshift plug-in so that on every message, if a key expected to be present in the metadata cache is not present, the plug-in checks for the environment variable. If the environment variable exists, the plug-in sets the metadata key to the value of the environment variable. Otherwise, it checks again on the next message. As a result, new gears from a scaled application may report a NULL namespace a few times until the OPENSHIFT_NAMESPACE environment variable is defined. Additionally, any other environment variables in the template will have the same checks. Note that all environment variables specified in the template are expected to be present for all applications. If an environment variable is specified that does not exist for an application, a slight performance degradation may be seen as the environment variable will be checked for on every message.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-07-21 15:12:06 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1566723 None None None Never
Red Hat Product Errata RHBA-2015:1463 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 2.2.6 bug fix and enhancement update 2015-07-21 19:11:33 EDT

  None (edit)
Description Josep 'Pep' Turro Mauri 2015-03-17 05:28:23 EDT
Description of problem:

mmopenshift fails to log namespace information for additional gears added after user varialbes exist in an application.

Version-Release number of selected component (if applicable):
rubygem-openshift-origin-controller-1.31.5.1-1.el6op.noarch
rsyslog7-mmopenshift-7.4.10-3.el6op.x86_64

How reproducible:
Always

Steps to Reproduce:
0.- configure mmopenshift with defaults as per the Admin Guide

1.- create a scalable app
    rhc app create -s myapp ruby-1.9

2.- add a custom env var to it
    rhc set-env MYVAR=something -a myapp

3.- add new gears to the app. Either by scaling it up:
    rhc cartridge scale -a myapp ruby-1.9 2

    or add a DB:
    rhc cartridge add -a myapp mysql-5.5

Actual results:
rsyslog messages coming from the new gears lack namespace information (ns is empty)

  2015-03-17T05:02:48.264634-04:00 vm ruby[22631]: app=myapp ns= appUuid=5507ed80e659c552f8000022 gearUuid=5507edaae659c552f8000045 [Tue Mar 17 05:02:48 2015] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations

  2015-03-17T05:07:59.752681-04:00 vm mysql[26778]: app=myapp ns= appUuid=5507ed80e659c552f8000022 gearUuid=5507eedde659c552f8000055 150317  5:07:59 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections.

Expected results:
New gears should log namespace details

Additional info:

When an application has user variables, Application#calculate_ops adds a PatchUserEnvVarsOp that pushes them to the new gear via rsync when OPENSHIFT_NAMESPACE is still not available in the new gear's env. This rsync triggers a log entry when the namespace information is not yet available inside the gear:

  Mar 17 05:07:49 vm rsyslogd-3000: mmopenshift: error opening file /var/lib/openshift/5507eedde659c552f8000055/.env/OPENSHIFT_NAMESPACE: No such file or directory

  2015-03-17T05:07:49.191439-04:00 vm oo-trap-user[26400]: app=myapp ns= appUuid=5507ed80e659c552f8000022 gearUuid=5507eedde659c552f8000055 oo_login: None from: 192.168.122.184 gearUuid=5507eedde659c552f8000055 cmd=rsync --server -prSe.iLs --delete . .env/user_vars

The available information for the gear is added to mmopenshift's gear info hash/cache (without namespace information) and all subsequent messages from that gear (uid) have an empty ns, until rsyslog is restarted (and information from gears has to be cached again).
Comment 2 Anping Li 2015-05-18 04:18:21 EDT
Verified and pass with the package in brew 
1) rpm package
[root@broker ~]# rpm -qa|grep syslog7
rsyslog7-mmopenshift-7.4.10-3.el6op.bz1202705.1.x86_64
rsyslog7-7.4.10-3.el6_6.x86_64
2) create scale application and scale up gear, and check the syslog message, the ns field are set correctly.

cat /var/log/openshift_gears

May 18 15:47:32 broker mysql[12388]: app=myapp ns=anlidom appUuid=555998714add7114b7000030 gearUuid=555999054add7114b7000065 150518 15:47:32 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections.
May 18 15:47:32 broker mysql[12388]: app=myapp ns=anlidom appUuid=555998714add7114b7000030 gearUuid=555999054add7114b7000065 150518 15:47:32 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections.

[root@broker ~]# cat /var/log/messages|grep "ns= "
[root@broker ~]#
Comment 6 errata-xmlrpc 2015-07-21 15:12:06 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1463.html

Note You need to log in before you can comment on or make changes to this bug.