Red Hat Bugzilla – Bug 1559433
Non participating nodes should not send rebalance data for a volume to graphite
Last modified: 2018-09-04 03:03:21 EDT
Description of problem: Currently in tendrl rebalance details get populated for non participating nodes as well for volumes. Only nodes participating in the volume should populate this data in graphite. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Could you provide more details about: * Which path in graphite is involved, where should we look? Is this path correct? "target": "tendrl.clusters.$cluster_id.volumes.$volume_name.rebal_status" Or do we need to check more paths?
I read the first comment once again and the question is more like: Am I right that there should not be present hosts, which are not participating for the volume, at "tendrl.clusters.$cluster_id.volumes.$volume_name.nodes" in graphite?
Lubos, yes you are correct. So if a node doesn't participate in a volume ideally no data should be pushed to graphite for the node under that volume in the path as you mentioned above. This causes invalid entries in grafana and while cluster -> volume -> nodes drill down you see all the nodes present in the list (even the ones which dont have bricks for the volume). Because of this if you try to look at bricks for volume the invalid node would have all invalid NA details. Hope this makes clear.
If there are no volume bricks on some node in cluster then during import there is not created `tendrl.clusters.$cluster_id.volumes.$volume_name.nodes.$node` target. If all bricks are deleted from a node that is managed in WA then targets `tendrl.clusters.$cluster_id.volumes.$volume_name.nodes.$node.bricks.$deleted_brick` are deleted and data is stopped to be pushed into `tendrl.clusters.$cluster_id.volumes.$volume_name.nodes.$node.rebalance_[bytes, failures, files, skipped]` (but endpoints remain with historical data). --> VERIFIED Tested with: tendrl-ansible-1.6.3-5.el7rhgs.noarch tendrl-api-1.6.3-4.el7rhgs.noarch tendrl-api-httpd-1.6.3-4.el7rhgs.noarch tendrl-commons-1.6.3-9.el7rhgs.noarch tendrl-grafana-plugins-1.6.3-7.el7rhgs.noarch tendrl-grafana-selinux-1.5.4-2.el7rhgs.noarch tendrl-monitoring-integration-1.6.3-7.el7rhgs.noarch tendrl-node-agent-1.6.3-9.el7rhgs.noarch tendrl-notifier-1.6.3-4.el7rhgs.noarch tendrl-selinux-1.5.4-2.el7rhgs.noarch tendrl-ui-1.6.3-8.el7rhgs.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2616