Bug 1470532

Summary: [RFE] Collect gluster metrics from nodes
Product: [oVirt] ovirt-engine-metrics Reporter: Sahina Bose <sabose>
Component: RFEsAssignee: Shirly Radco <sradco>
Status: CLOSED DEFERRED QA Contact: Lukas Svaty <lsvaty>
Severity: medium Docs Contact:
Priority: medium    
Version: unspecifiedCC: bugs, seamurph, sradco
Target Milestone: ---Keywords: FutureFeature
Target Release: ---Flags: sbonazzo: ovirt-4.3-
lsvaty: testing_plan_complete+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-19 08:27:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Gluster RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1559864    
Bug Blocks:    

Description Sahina Bose 2017-07-13 06:41:48 UTC
Description of problem:

We would like to collect gluster related metrics from each node to help diagnose issues/performance better
Some metrics that are likely to be interesting
- Brick 
  -- capacity utilization
  -- IO stats
  -- entries to be healed
  -- number of connections
- Gluster process - CPU utilization
- Number of bricks and status
- Status of gluster processes

Version-Release number of selected component (if applicable):


How reproducible:
NA

Comment 1 Sahina Bose 2017-07-13 06:44:24 UTC
Sean, could you add to this?

Comment 2 Sahina Bose 2017-08-22 09:25:32 UTC
Are all these metrics collected by the Tendrl plugin

Comment 3 anmol babu 2017-08-22 09:32:32 UTC
- Brick 
  -- capacity utilization -- Yes available in tendrl collectd plugin
  -- IO stats -- Yes available in tendrl collectd plugin
  -- entries to be healed -- Yes available in tendrl collectd plugin
  -- number of connections -- Yes available in tendrl collectd plugin
- Gluster process - CPU utilization
      No process wise cpu consumption monitored currently.
      We have net cpu utilization consumed by various categories like system processes, user processes etc...
- Number of bricks and status -- Yes available in tendrl collectd plugin
- Status of gluster processes -- Not available currently

Additionally the detailed list of metrics that tendrl currently has can be found at: https://github.com/Tendrl/documentation/wiki/Metrics

Comment 4 Shirly Radco 2017-10-23 10:25:08 UTC
IIUC the link specifies the collectd plugins that will also report gluster stats.
I did not see a "tendrl collectd plugin" that requires installation and configuration on collectd side. Is that correct?

Comment 9 Venkata R Edara 2018-03-07 06:54:23 UTC
we decided to make separate repository for gluster-collectd plugin. we rasied bz to create repository under gluster.org. The upstream version of collectd is based on C language and we have gluster plugin based on python. to maintain the plugin in better way we decided to have seperate repository and package it. 

once the repository is created, we will ship the rpms to fedora and centos and will have ansible playbook to install it.

Comment 10 Shirly Radco 2018-03-07 07:00:37 UTC
(In reply to Venkata R Edara from comment #9)
> we decided to make separate repository for gluster-collectd plugin. we
> rasied bz to create repository under gluster.org. The upstream version of
> collectd is based on C language and we have gluster plugin based on python.
> to maintain the plugin in better way we decided to have seperate repository
> and package it. 
> 
> once the repository is created, we will ship the rpms to fedora and centos
> and will have ansible playbook to install it.

I don't think it should be in a different ansible playbook.
If its available in u/s and d/s we can add it to the ovirt-engine-metrics repo.

Comment 11 Venkata R Edara 2018-03-07 07:08:08 UTC
(In reply to Shirly Radco from comment #10)
> (In reply to Venkata R Edara from comment #9)
> > we decided to make separate repository for gluster-collectd plugin. we
> > rasied bz to create repository under gluster.org. The upstream version of
> > collectd is based on C language and we have gluster plugin based on python.
> > to maintain the plugin in better way we decided to have seperate repository
> > and package it. 
> > 
> > once the repository is created, we will ship the rpms to fedora and centos
> > and will have ansible playbook to install it.
> 
> I don't think it should be in a different ansible playbook.
> If its available in u/s and d/s we can add it to the ovirt-engine-metrics
> repo.

Sure. the playbook wont be in different repo. we can add it to ovirt-engine-metrics repo. we will notify once the playbook is ready.

Comment 12 Shirly Radco 2018-08-30 12:32:16 UTC
Hi, Any updates?

Comment 13 Venkata R Edara 2018-09-17 13:23:36 UTC
The playbooks are ready. we are shipping 2 packages. gluster-collectd plugin which is to be installed on ovirt nodes where gluster is installed. one more package is choose-master which uses ovirt sdk api to choose ovirt nodes which are up and chooses one node to get volume metrics. 

on ovirt engine, install this rpm
https://copr-be.cloud.fedoraproject.org/results/redara/gluster-collectd/fedora-27-x86_64/00798077-choose-master/choose-master-1.0.0-1.fc27.noarch.rpm

on ovirt nodes install rpm
https://copr-be.cloud.fedoraproject.org/results/redara/gluster-collectd/fedora-27-x86_64/00795761-gluster-collectd/gluster-collectd-1.0.0-1.fc27.noarch.rpm

playbook:
https://github.com/gluster/gluster-collectd/pull/2

we will add more documentation on how to use playbook.

Comment 15 Sandro Bonazzola 2019-01-28 09:40:23 UTC
This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

Comment 16 Sahina Bose 2019-07-19 08:27:52 UTC
Unlikely that this is going to be integrated. Deferring this