Bug 1531561

Summary: [RFE][RADAR] New Metering Calculation for Middleware Products running on OpenShift
Product: Red Hat CloudForms Management Engine Reporter: Loic Avenel <lavenel>
Component: ReportingAssignee: Gregg Tanzillo <gtanzill>
Status: CLOSED CURRENTRELEASE QA Contact: Dave Johnson <dajohnso>
Severity: high Docs Contact:
Priority: high    
Version: 5.9.0CC: cpelland, dozdowsk, gtanzill, jalberts, obarenbo
Target Milestone: GAKeywords: FutureFeature, TestOnly
Target Release: 5.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.10.0.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1539074 (view as bug list) Environment:
Last Closed: 2018-06-21 20:22:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Feature
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1539074    

Description Loic Avenel 2018-01-05 14:12:41 UTC
Description of requirements: New Metering Calculation for Middleware Products running on OpenShift

Comment 2 dozdowsk 2018-01-05 17:16:52 UTC
Filing / continuation of discussions from before shutdown with Oleg and Greg.

PRIMARY METERING ALGORITHM

Per universal clock our, report the maximal value for instantaneous CPU core usage across containers bearing a given label inside OpenShift.  Visual Aid - https://docs.google.com/presentation/d/13O1vQW_gJR_S4L16C9T2XOnz0xp0gzJInR8EkTo3JBQ/edit#slide=id.g27cdf96659_2_480

SIGNIFICANT REPORTING CONCERNS

Per label the report should be capable of grouping by project, sorting usage of common images into "buckets." Customers will use these groupings to index use against cost centers requesting and operating the projects.

Assignation of a literal rate to this usage, integrating with chargeback features, is considered a nicety - but not essential for subscription reporting needs.

ADDITIONAL REPORTING CONCERNS

The middleware business unit is in discussions with various parties to use memory as the active metric for calculating subscription "size."  The ability to map the "hourly instantaneous maximum" calculation against memory usage would be an extremely wise feature to build in.

Further, the middleware business unit is contemplating the use of allocation rather than literal usage for calculating subscription "size."  The ability to map the "hourly instantaneous maximum" calculation against allocation of both memory AND core usage would be an excellent addition, if possible.

DATA RETENTION

The middleware business unit is seeking feedback from engineering regarding what is possible for data retention.  Primary use of retained data would be explaining or justifying an hourly value/calculation to a customer.  Ideal data retention would preserve all sampled data for a relatively short period of time, while holding a condensed "rollup" for a longer period.  

The rollup can be as simple as "For the 1400-1500 hour on January 23, 2018 the calculated maximum was 24 cores at 1437.  Pod 1 from image X = 2.4 cores.  Pod 2 from image X = 2.6 cores.  Pod 1 from image Y = 7 cores.  Pod 2 from image Y = 7.4 cores.  Pod 3 from image Y = 2.1 cores.  Pod 1 from image Z = 2.2 cores."

Comment 3 Gregg Tanzillo 2018-01-11 16:30:12 UTC
Dan, do you have a live environment that we can use to construct and verify the report are we work on it? This will help us tremendously so the we can focus on the building the report and not lose time trying to build an env and collect data.

Comment 4 dozdowsk 2018-01-11 16:35:11 UTC
Hopefully this will do for you:

https://docs.google.com/document/d/1US6GiBsRqJ4syJGL_cCQAZr3pzu8U83eaecwlVznYpU/edit

Doc should be viewable to you.  Ansible QE stood this up for us - credit where due.  :)

Comment 7 CFME Bot 2018-01-19 16:17:25 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/4ae0dfe26f93c7947158dfbdad9a0352878a746b

commit 4ae0dfe26f93c7947158dfbdad9a0352878a746b
Author:     Gregg Tanzillo <gtanzill>
AuthorDate: Fri Jan 19 06:48:31 2018 -0500
Commit:     Gregg Tanzillo <gtanzill>
CommitDate: Fri Jan 19 06:48:31 2018 -0500

    Move `sqlite3` into main section of Gemfile
    Need Sqlite for some new tools currently under development
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1531561

 Gemfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comment 8 CFME Bot 2018-01-25 16:37:32 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/8969f7b27d1b471c2bd58bfb1abbc6fba1ee2308

commit 8969f7b27d1b471c2bd58bfb1abbc6fba1ee2308
Author:     Gregg Tanzillo <gtanzill>
AuthorDate: Tue Jan 16 07:01:54 2018 -0500
Commit:     lpichler <lpichler>
CommitDate: Wed Jan 17 14:29:55 2018 +0100

    Version 1 capture script for radar project
    
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1531561

 tools/capture_radar.rb | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
 create mode 100644 tools/capture_radar.rb

Comment 9 CFME Bot 2018-01-25 16:37:42 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/fd6bce26bff4713a472980b367a46a26fcbd37b2

commit fd6bce26bff4713a472980b367a46a26fcbd37b2
Author:     Gregg Tanzillo <gtanzill>
AuthorDate: Tue Jan 16 08:07:26 2018 -0500
Commit:     lpichler <lpichler>
CommitDate: Wed Jan 17 14:29:55 2018 +0100

    Added container project
    Metric rollups are separated by timestamp, CustomAttribute (label) and project name (resource_name)
    
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1531561

 tools/capture_radar.rb | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

Comment 10 CFME Bot 2018-01-25 16:37:46 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/676dc0160dfee24db1a7244f06db24dac49b8982

commit 676dc0160dfee24db1a7244f06db24dac49b8982
Author:     Gregg Tanzillo <gtanzill>
AuthorDate: Tue Jan 16 07:41:18 2018 -0500
Commit:     lpichler <lpichler>
CommitDate: Wed Jan 17 14:29:55 2018 +0100

    Persist max CPU cores used by label as MetricRollup
    
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1531561

 tools/capture_radar.rb | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)