Bug 1081900
| Summary: | [Nagios] [RFE] Alerting mechanism for split-brain from Nagios | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Prasanth <pprakash> |
| Component: | gluster-nagios-addons | Assignee: | Sahina Bose <sabose> |
| Status: | CLOSED ERRATA | QA Contact: | RamaKasturi <knarra> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | rhgs-3.0 | CC: | annair, asrivast, divya, dpati, knarra, nlevinki, nsathyan, rhs-bugs, sabose, sdharane, ssaha |
| Target Milestone: | --- | Keywords: | FutureFeature, Reopened |
| Target Release: | RHGS 3.1.0 | Flags: | divya:
needinfo+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | gluster-nagios-addons-0.2.1-1 | Doc Type: | Enhancement |
| Doc Text: |
Previously, there was no way to alert the user when split-brain is detected on a replicate volume. Due to this, users did not know the issue to take timely corrective action. With this enhancement, the Nagios plugin for self-heal monitoring has been enhanced to report if any of the entries are in split-brain state. Plugin has been renamed from "Volume Self-heal" to "Volume Split-brain status".
|
Story Points: | --- |
| Clone Of: | 1033197 | Environment: | |
| Last Closed: | 2015-07-29 05:25:34 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1100563 | ||
| Bug Blocks: | 1033197, 1202842 | ||
|
Description
Prasanth
2014-03-28 07:28:22 UTC
*** This bug has been marked as a duplicate of bug 1033197 *** Not sure why this bug was closed as duplicate as it was created specifically for having the feature included in Nagios as per the last Bug triage: ----------------------------------------------- As discussed in the triage meeting, a new bug is now opened to track this feature through Nagios. ( Currently Alerts would not be shown in RHSC. They will be shown only in Nagios UI ) ---------Note from triage meeting-------------- 1033197 - Out, for now. A different bug will be created for monitoring split-brain using Nagios. (Bug 1081900 opened for the same) ----------------------------------------------- Hence re-opening it. Currently there is way in gluster to identify a split brain and so in Nagios UI there is no way to alert the case of a split brain. Currently in Nagios the split brain scenario is being identified based on the quorum check for the volume. Small correction in the comment earlier. Please read as below - "Currently there is NO way in gluster to identify a split brain and so in Nagios UI there is no way to alert the case of a split brain. Currently in Nagios the split brain scenario is being identified based on the quorum check for the volume." Sorry for the typo. As discussed with Alok, Vijay and other key stake holders over e-mail, i am taking this bug out of Denali release. We will be taking the following in for Everglades: 1. Alerting when files are in split brain (using the "gluster volume heal split-brain info") 2. When there's a network split-brain this is currently alerted using the Cluster-quorum plugin (this plugin will alert the administrator when volumes have lost quorum as long as server side quorum is turned on) Patches http://review.gluster.org/9782 and http://review.gluster.org/9783 posted Verified and works fine with gluster-nagios-addons-0.2.3-1.el6rhs.x86_64. Currently when nagios detects that split brain has occurred it marks the Volume Split-Brain status - <vol_name> service to critical and shows how many no.of files are in split brain. When there is no split brain detected, Volume Split-brain status - <vol_name> remains in OK state with status information as "No split brain state entries found". When the volume is stopped / deleted, Volume Split-brain status - <vol_name> displays the status as WARNING with status information as "split brain status could not be determined" An email and snmp notifications are sent when split brain status changes to critical and when it comes back to normal again. Sahina, Please review the edited doc text and sign-off. Acked Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-1494.html |