| Summary: | pcp pmwebd performance improvements | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Frank Ch. Eigler <fche> |
| Component: | pcp | Assignee: | pcp-maint <pcp-maint> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 26 | CC: | brolley, fche, lberk, mgoodwin, nathans, pcp, will_darton, zcerza |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | 3.12.2-1 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-10-25 21:35:05 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle. Changing version to '26'. Frank, how much of this is in mainline and/or your pcpfans tree, what's left to be done? (In reply to Mark Goodwin from comment #2) > Frank, how much of this is in mainline and/or your pcpfans tree, what's left > to be done? Frank is on PTO for a few weeks Mark. Nothing has been sent upstream at this stage, although of course Ken's libpcp threading improvements continue onward independently. pmfuntools pmwebd -J related functionality is in pcp 3.12.2. |
pmwebd is slow to answer certain types of queries, sometimes painfully so. Some ideas: - fix multithreading problems in libpcp, so that the core pmwebapi portion of pmwebd could become multithreaded - fix multithreading problems in libpcp, so that the graphite pmwebapi portion can be fully multithreaded too (with concurrent service of different requests, rather than just concurrent scans of multiple archives for a given request) - for graphite mode, cache archive metadata somehow. We can't keep all of the archives open because of FD limits. Plus we must be tolerant to archives that are being actively written-to (so the metadata cache must be flushable). - for graphite-json query outputs, investigate whether all the JSON "{null, $timestamp}" entries actually must be emitted. Maybe the graphite/grafana webapps we bundle can fill in that stuff. - for graphite-png query outputs, rework the representation of all the timeseries in memory as a big rectangular vector-of-vectors. The nulls/nans probably don't need to be materialized, and costing perhaps tens or hundreds of MB (for massive sets of series).