Bug 1879350

Summary: [RFE] Provide a pcp-ss(1) tool in pcp-system-tools
Product: Red Hat Enterprise Linux 8 Reporter: Yogesh Babar <ybabar>
Component: pcpAssignee: Mark Goodwin <mgoodwin>
Status: CLOSED ERRATA QA Contact: Jan Kurik <jkurik>
Severity: high Docs Contact: Petr Hybl <phybl>
Priority: unspecified    
Version: 8.0CC: agerstmayr, igkioka, jkurik, jvaldez, mgoodwin, nathans, patrickm
Target Milestone: rcKeywords: FutureFeature, Triaged
Target Release: 8.5   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: pcp-5.3.1-3.el8 Doc Type: Enhancement
Doc Text:
.The new `pcp-ss` PCP utility is now available The `pcp-ss` PCP utility reports socket statistics collected by the `pmdasockets(1)` PMDA. The command is compatible with many of the `ss` command line options and reporting formats. It also offers the advantages of local or remote monitoring in live mode and historical replay from a previously recorded PCP archive.
Story Points: ---
Clone Of:
: 1981223 (view as bug list) Environment:
Last Closed: 2021-11-09 17:49:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1981223    

Description Yogesh Babar 2020-09-16 03:42:07 UTC
Description of problem:

We do not have 'pcp SS' tool in our PCP inventory. 

Version-Release number of selected component (if applicable):

pcp-5.0.2-5.el8.x86_64
pcp-system-tools-5.0.2-5.el8.x86_64

How reproducible:
N/A

Actual results:
N/A

Expected results:

The 'pcp SS' should be inspired by the OS shipped 'SS' command and it should support almost all the features (flags) provided by SS. 


Additional info:

Our pcp-system-tool does provide tools like 'pcp ipcs, pcp free, pcp iostat, pcp atop' etc and they really help support team in analyzing the performance issue from CPU, Memory & I/O. But we do not have similar tool to monitor the network resources. Surely we can write our own custom tool with pmrep.conf but it seems we need to add many additional metrics to match the SS output. 

We in support have lot of cases where historical data captured by pmlogger of SS & netstat would help us in resolving the performance cases. 

So please consider this RFE. 

Let me know if any additional information is required. 

Regards,
Ybabar

Comment 3 Mark Goodwin 2021-03-09 02:47:55 UTC
Setting DTM to 18, which is a few weeks before BZ#1922040 (Rebase PCP to 5.3.0+)

Comment 4 Mark Goodwin 2021-06-24 23:35:05 UTC
Committed upstream for pcp-5.3.2:

commit d2e9fd0dce7f24b8f81109b2b664ca637abfd2fc
Author: Mark Goodwin <mgoodwin>
Date:   Thu Jun 24 12:42:54 2021 +1000

    pcp-ss: initial version of new client tool, with QA and docs
    
    Initial commit for pcp-ss(1) front end client for pmdasockets(1),
    implementing many (but not yet all) of the ss(8) options. This
    is packaged in pcp-system-tools. New QA test 1633 is added along
    with test archive and man page.
    
    Still TODO are some PMDA changes for a persistent filter config
    file and various additional metrics, e.g. supporting the --process
    option, see https://github.com/performancecopilot/pcp/issues/1231
    
    Resolves: RHBZ#1879350


New QA test 1633 can be used to verify.

Comment 5 Mark Goodwin 2021-06-29 01:34:23 UTC
Upstream commits in addition to those in Comment#4:

commit 47f9df215a609c5b3097080e22a1415ad83e2b8e
Author: Mark Goodwin <mgoodwin>
Date:   Mon Jun 28 16:44:40 2021 +1000

    pcp-ss: fix EAGAIN exceptions, add filter header, QA
    
    Use explicit scale="instant" in fetchgroups to avoid rate conversion.
    This was causing fetchgroups for metrics with counter semantics
    to throw EAGAIN exceptions in the report function. pcp-ss reports
    the raw counters, but other tools such as pmrep want to do rate
    conversion, so we keep the correct (native) semantics.
    
    Also simplify the code a bit, add missing bytes_sent in the --info
    report. Add a header in the report for the current timestamp and
    filter, which is especially useful for archive replay since the
    filter can change over time whilst logging (it's a storable metric).
    
    Tweaked pcp-ss report in qa/1634.out

commit 6e19b22d183b3a5118ab9f5e0d1b4e7bc8a73838
Author: Mark Goodwin <mgoodwin>
Date:   Mon Jun 28 16:20:31 2021 +1000

    pmdasockets: add persistent filter, new metrics, parser fixes and QA
    
    Add persistent filter (loaded on start only) as a config file
    in PCP_SYSCONF_DIR/sockets/filter.conf This can be overridden
    at runtime with a store to network.persocket.filter.
    
    Tweak the parser to handle the case where some filters (e.g.
    "state established") cause the second column in the ss report
    to be omitted. Also add some appl0 and app1 debug diags.
    
    qa/1634 updated to use the QA domain (FORQA=251) and a private
    pmns, avoiding cached indom updates by pmlogger or other clients
    whilst the test is running (was occuring when the sockets PMDA
    is enabled, causing intermittent mysterious timing related
    failures)
    
    Related: RHBZ#1879350 - pcp-ss

commit af2c47e7755dc27db145f3568ead61458181ccdf
Author: Marko Myllynen <myllynen>
Date:   Thu Jun 24 13:06:29 2021 +0300

    pcp-ss.1: fix syntax, sort SEE ALSO
    
    Also drop some unrelated references like pmchart(1).

Comment 6 Mark Goodwin 2021-06-29 04:27:39 UTC
Couple more:

commit 1ff17fc3220a5d4bf3980f0fe5bef44ba29be7bc (HEAD -> main)
Author: Mark Goodwin <mgoodwin>
Date:   Tue Jun 29 14:24:57 2021 +1000

    docs: document pmdasockets filter config file
    
    Updated pmdasockets(1) man page.

commit a2a4527d328b6c737c744f3b58fd6012616bd9bc
Author: Mark Goodwin <mgoodwin>
Date:   Tue Jun 29 14:00:28 2021 +1000

    pmdasockets: fix resource leak coverity CID371684
    
    Plug leak in error handling path:

Comment 7 Mark Goodwin 2021-07-09 08:21:43 UTC
Note to QE: the fix for  BZ#1980459 - "Test '000' of the pcp upstream testsuite fails if hacluster pmda is installed and registered" did not make it into the pcp-5.3.1-3 build. So test 000 will fail if either of the sockets or hacluster PMDA is enabled (they are both optional PMDA and not enabled by default).

Comment 10 Jan Kurik 2021-07-14 04:13:26 UTC
Switching to VERIFIED as the tool is implemented and works.
Follow-up issues are tracked in bz 1981886.

Comment 14 errata-xmlrpc 2021-11-09 17:49:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pcp bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4171