Bug 1415816

Summary: Integrating PCP's graphite emulation with Grafana is unusable
Product: Red Hat Enterprise Linux 7 Reporter: Peter Portante <pportant>
Component: pcpAssignee: Frank Ch. Eigler <fche>
Status: CLOSED ERRATA QA Contact: Martin Cermak <mcermak>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: brolley, brubisch, cww, fche, lberk, mbenitez, mcermak, mgoodwin, myllynen, nathans, sreber
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: pcp-3.12.2-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 17:04:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1420851    
Attachments:
Description Flags
test logs none

Description Peter Portante 2017-01-23 19:29:31 UTC
We have a 28 node environment where is node has PCP metrics send to a central server.

When I run the graphite emulation service for PCP archives and create a Grafana v4 data source for that instance, I find that it takes many minutes for the metrics selection widget to populate the available metrics.

This renders the integration unusable at this small scale.

Comment 3 Frank Ch. Eigler 2017-06-23 19:55:44 UTC
Please consider taking a look at the version of pmwebd in this COPR.  It aims to address this and related problems.

https://copr.fedorainfracloud.org/coprs/fche/pcp/

Comment 4 Simon Reber 2017-06-28 09:47:44 UTC
(In reply to Frank Ch. Eigler from comment #3)
> Please consider taking a look at the version of pmwebd in this COPR.  It
> aims to address this and related problems.
> 
> https://copr.fedorainfracloud.org/coprs/fche/pcp/
Would you have information available about the changes included? It would be nice to have some information available how to use the packages, knowing the changes implemented and if there is any difference in the usage.

Comment 5 Simon Reber 2017-06-28 09:48:23 UTC
(In reply to Frank Ch. Eigler from comment #3)
> Please consider taking a look at the version of pmwebd in this COPR.  It
> aims to address this and related problems.
> 
> https://copr.fedorainfracloud.org/coprs/fche/pcp/
Would you have information available about the changes included? It would be nice to have some information available how to use the packages, knowing the changes implemented and if there is any difference in the usage.

Comment 6 Frank Ch. Eigler 2017-06-30 14:17:07 UTC
> > https://copr.fedorainfracloud.org/coprs/fche/pcp/
> Would you have information available about the changes included? 

Certainly.  The build comes from git://sourceware.org/git/pcpfans.git branch fche/pmfuntools, which is regularly rebased on top of pcp master, as long as we have features that have not yet been merged upstream.

The big ticket item is "pmwebd -J", a new default option that dramatically speeds up pmwebd when processing graphite queries against a potentially huge set of archives, and allows representation of a barely-mangled hostname (instead of a highly mangled archive path name) as the first component of the graphite metrics.

From [man pmwebd]:

       -J     When constructing graphite metric names, use the stored hostname instead of a archive pathname as the first component.  This virtually unifies all archives found with the same hostname into a single time series.  The host name is canonicalized: any symbol characters other than _ (underscore), space, - (hyphen), and / (slash) are replaced by _ (underscore).

Comment 12 Martin Cermak 2018-03-15 18:14:40 UTC
Created attachment 1408516 [details]
test logs

Comment 13 Martin Cermak 2018-03-15 18:16:21 UTC
Verified with pcp-3.12.2-5.el7.

Comment 16 errata-xmlrpc 2018-04-10 17:04:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0926