Bug 1373123

Summary: monitor OpenShift with NewRelic
Product: OpenShift Container Platform Reporter: Miheer Salunke <misalunk>
Component: RFEAssignee: Jeff Cantrill <jcantril>
Status: CLOSED DUPLICATE QA Contact: Johnny Liu <jialiu>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.3.1CC: amurdaca, aos-bugs, dwalsh, erich, jokerman, lpioro, lsm5, mmccomas
Target Milestone: ---Keywords: Extras
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-16 16:27:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Miheer Salunke 2016-09-05 09:50:12 UTC
Description of problem:

On OpenShift Enterprise environment - I am trying to enable the NewRelic monitoring of docker as per
https://docs.newrelic.com/docs/servers/new-relic-servers-linux/installation-configuration/enabling-new-relic-servers-docker

I tried to add newrelic use to both 'docker' and 'dockerroot' groups, but none of these help - the docker metrics are not seen in New Relic.

When I did this with in the past with 'pure' docker [no OpenShift] containers, metrics were seen in New Relic OK.





Version-Release number of selected component (if applicable):


cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)

nrsysmond -v
New Relic system monitor daemon version 2.3.0.132 ("cloudy") ("8cff29599e1e9619b31da739b2ac86c7213f6b71")
(C) Copyright 2009-2016 New Relic Inc. All rights reserved.
curl version libcurl/7.42.1 OpenSSL/1.0.1t zlib/1.2.8
pcre version 8.36 2014-09-26
zlib version 1.2.8

oc version
oc v3.2.1.13-1-gc2a90e1
kubernetes v1.2.0-36-g4a3f9c5

docker version
Client:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-common-1.10.3-46.el7.10.x86_64
 Go version:      go1.6.2
 Git commit:      2a93377-unsupported
 Built:           Fri Jul 29 13:45:25 2016
 OS/Arch:         linux/amd64

Server:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-common-1.10.3-46.el7.10.x86_64
 Go version:      go1.6.2
 Git commit:      2a93377-unsupported
 Built:           Fri Jul 29 13:45:25 2016
 OS/Arch:         linux/amd64




How reproducible:
always, no matter if I try OpenShift Enterprise v3.2.1.13-1-gc2a90e1 or Origin v1.2.1

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 8 Daniel Walsh 2016-09-09 17:55:53 UTC
Seems like multiple problems here.  One is the socket not being owned by the docker group.  I don't know why this is not happening on RHEL, it does work on Fedora.  Second problem is the docker does not seem to be responding in json data.

BTW Adding the user to the root group versus the docker group is pretty much the same from a security point of view.

Once you can write to the docker.sock as non root you pretty much control the system.

http://www.projectatomic.io/blog/2015/08/why-we-dont-let-non-root-users-run-docker-in-centos-fedora-or-rhel/

Comment 9 Daniel Walsh 2016-09-09 17:58:21 UTC

Antonio wasn't there a bug where docker info was not returning json data?

Comment 15 Miheer Salunke 2016-09-14 14:55:28 UTC
Hello,  below you shall find a list of steps to reproduce:

1] install latest oc enterprise [v3.2.1.13-1-gc2a90e1]

2] register a NewRelic account [free trial is perfectly enough] here https://newrelic.com/

3] on any of your oc servers, perform deploy New Relic Server agent:
a] cat << EOF > /etc/yum.repos.d/newrelic.repo
[newrelic]
name=New Relic packages for Enterprise Linux - $basearch
baseurl=http://yum.newrelic.com/pub/newrelic/el5/$basearch
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-NewRelic
EOF
b] yum install -y newrelic-sysmond
c] vim /etc/newrelic/nrsysmond.cfg
  - enter the license_key which is linked to your New Relic account [seen in menu / account settings as "License key"] 
  - change loglevel - preferably to verbosedebug to see max  [log is in /var/log/newrelic/nrsysmond.log]
d] systemctl restart newrelic-sysmond

4] go to New Relic account / Servers / see your server details in few minutes - what is expected is to see an extra 'docker' menu on left side with docker-related info

and last important step is to enable NewRelic agent as:

https://docs.newrelic.com/docs/servers/new-relic-servers-linux/installation-configuration/enabling-new-relic-servers-docker


please let me know if you need any additional info / details, thanks a lot for coop / update!

Comment 21 Miheer Salunke 2016-09-16 16:27:33 UTC
After doing 'yum upgrade docker' to docker-common-1.10.3-46.el7.14.x86_64.rpm all is fine now, we can monitor docker statistics via New Relic agent.

So closing this BZ.

Solved by  https://rhn.redhat.com/errata/RHBA-2016-1827.html

Comment 22 Miheer Salunke 2016-09-16 16:39:08 UTC

*** This bug has been marked as a duplicate of bug 1368999 ***