Bug 1559433 - Non participating nodes should not send rebalance data for a volume to graphite
Summary: Non participating nodes should not send rebalance data for a volume to graphite
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: web-admin-tendrl-node-agent
Version: rhgs-3.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.4.0
Assignee: gowtham
QA Contact: Filip Balák
URL:
Whiteboard:
Depends On:
Blocks: 1503137
TreeView+ depends on / blocked
 
Reported: 2018-03-22 14:31 UTC by Shubhendu Tripathi
Modified: 2018-09-04 07:03 UTC (History)
5 users (show)

Fixed In Version: tendrl-node-agent-1.6.1-3.el7rhgs
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-09-04 07:02:28 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github Tendrl node-agent issues 730 0 None None None 2018-04-03 10:35:55 UTC
Red Hat Bugzilla 1599691 0 unspecified CLOSED Volume data should not be sent to graphite from nodes with deleted bricks 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHSA-2018:2616 0 None None None 2018-09-04 07:03:21 UTC

Internal Links: 1599691

Description Shubhendu Tripathi 2018-03-22 14:31:22 UTC
Description of problem:
Currently in tendrl rebalance details get populated for non participating nodes as well for volumes. Only nodes participating in the volume should populate this data in graphite.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Martin Bukatovic 2018-03-28 10:04:11 UTC
Could you provide more details about:

* Which path in graphite is involved, where should we look?

Is this path correct?

"target": "tendrl.clusters.$cluster_id.volumes.$volume_name.rebal_status"

Or do we need to check more paths?

Comment 3 Lubos Trilety 2018-03-28 10:10:28 UTC
I read the first comment once again and the question is more like:
Am I right that there should not be present hosts, which are not participating for the volume, at "tendrl.clusters.$cluster_id.volumes.$volume_name.nodes" in graphite?

Comment 4 Shubhendu Tripathi 2018-03-28 11:37:42 UTC
Lubos, yes you are correct. So if a node doesn't participate in a volume ideally no data should be pushed to graphite for the node under that volume in the path as you mentioned above. This causes invalid entries in grafana and while cluster -> volume -> nodes drill down you see all the nodes present in the list (even the ones which dont have bricks for the volume). Because of this if you try to look at bricks for volume the invalid node would have all invalid NA details.

Hope this makes clear.

Comment 7 Filip Balák 2018-08-01 11:55:25 UTC
If there are no volume bricks on some node in cluster then during import there is not created `tendrl.clusters.$cluster_id.volumes.$volume_name.nodes.$node` target. If all bricks are deleted from a node that is managed in WA then targets `tendrl.clusters.$cluster_id.volumes.$volume_name.nodes.$node.bricks.$deleted_brick` are deleted and data is stopped to be pushed into `tendrl.clusters.$cluster_id.volumes.$volume_name.nodes.$node.rebalance_[bytes, failures, files, skipped]` (but endpoints remain with historical data). --> VERIFIED

Tested with:
tendrl-ansible-1.6.3-5.el7rhgs.noarch
tendrl-api-1.6.3-4.el7rhgs.noarch
tendrl-api-httpd-1.6.3-4.el7rhgs.noarch
tendrl-commons-1.6.3-9.el7rhgs.noarch
tendrl-grafana-plugins-1.6.3-7.el7rhgs.noarch
tendrl-grafana-selinux-1.5.4-2.el7rhgs.noarch
tendrl-monitoring-integration-1.6.3-7.el7rhgs.noarch
tendrl-node-agent-1.6.3-9.el7rhgs.noarch
tendrl-notifier-1.6.3-4.el7rhgs.noarch
tendrl-selinux-1.5.4-2.el7rhgs.noarch
tendrl-ui-1.6.3-8.el7rhgs.noarch

Comment 9 errata-xmlrpc 2018-09-04 07:02:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2616


Note You need to log in before you can comment on or make changes to this bug.