Red Hat Bugzilla – Bug 1559426
Sometimes monitoring-integration is not creating panels for a particular resource in alert dashbaord
Last modified: 2018-09-04 03:03:21 EDT
Description of problem: If panel for a particular resource is already present in alerting dashboard then monitoring-integration won't create a duplicate panel for that resource. While checking duplication monitoring-integration is matching substring also, Because of this any resource_name which match with some other resource_name's substring then the panel is not created for that new resource. (e.g) if panel name is v11 then if new panel name is v1 then v1 is matching with a substring of v11, So it considered as duplicate and alert panel is not created. Version-Release number of selected component (if applicable): How reproducible: Create volume with name v11 first and create volume with name v1 Steps to Reproduce: 1. 2. 3. Actual results: Volume_name if matches with existing volume_name substring then alert panel for a new volume is not created. Expected results: Alert dashbaord should have panels for all resources Additional info:
Could you provide more details about: * Is it possible to reproduce this on previously released RHGS WA? * Version where the bug is present. * Where should we look for the panes mentioned in BZ description in grafana exactly?
It can be reproducible in the previous release like 1.6.1-1, Actually, alert panels are hidden for the users, to check the alert dashbaord please sign in with credentials in grafana. Then you can see the two organization: 1. main org 2. Alert dashbaord org So we are using the second organization for raising alerts when threshold breach crosses 75%. For each resource, we are creating a separate alert panel for each resource. So while creating an alert panel we are comparing alert panel resource name with an already present alert panel resource name, problem is if new resource name match with already existing resource names substring then alert panel for new resource is not created. So no utilization alert will be raised for that resource. (e.g) if volume_alert_dashboard have panels for volume V11 then if new volume is created with name V1 then alert panel for V1 is not created because V1 is matching with substring V11. So no utilization alerts raised for V1. To see alert dashbaord: 1. Do sign-in with grafana 2. Then you can see new see a new organization with name Alert dashboard 3. Switch the control to Alert dashboard 4. press Home to see list of alert dashbaords
In version 3.3.1 it can be reproducible
I'm assuming that alert dashboard in grafana is mentioned only as a hint for qe team and this feature is still not supported (as this is not documented, there is no feature BZ for it and my understanding was that this is internal implementation details to support alerts[1]) [1] see eg. this note from Mrugesh: > Alerts org will contain the panels created for alert callbacks and will be > hidden from the end users. from https://github.com/Tendrl/specifications/issues/191#issuecomment-326197800 Is this correct?
That's correct. But if alert dashboard is not created properly, grafana will not send any alerts to tendrl for any thing related to cpu, memory etc threshold chnages and will not reflect in the alert panel
Seems ok. --> VERIFIED Tested with: tendrl-ansible-1.6.3-4.el7rhgs.noarch tendrl-api-1.6.3-3.el7rhgs.noarch tendrl-api-httpd-1.6.3-3.el7rhgs.noarch tendrl-commons-1.6.3-5.el7rhgs.noarch tendrl-grafana-plugins-1.6.3-3.el7rhgs.noarch tendrl-grafana-selinux-1.5.4-2.el7rhgs.noarch tendrl-monitoring-integration-1.6.3-3.el7rhgs.noarch tendrl-node-agent-1.6.3-5.el7rhgs.noarch tendrl-notifier-1.6.3-3.el7rhgs.noarch tendrl-selinux-1.5.4-2.el7rhgs.noarch tendrl-ui-1.6.3-2.el7rhgs.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2616