Bug 1461129

Summary:	malformed cluster.server-quorum-ratio setting can lead to split brain
Product:	[Community] GlusterFS	Reporter:	Michael Adam <madam>
Component:	glusterd	Assignee:	bugs <bugs>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	amukherj, bugs
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.12.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-09-05 17:34:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Michael Adam 2017-06-13 14:49:28 UTC

Description of problem:

cluster.server-quorum-ratio takes a percentage value.
If a value is put into it which is not a valid
percentage value, Glusterd will treat "0" nodes
as sufficient for quorum, hence possibly leading
to split brain.


Version-Release number of selected component (if applicable):

Any.

How reproducible:


Steps to Reproduce:
1. set up a volume and configure server quorum
2. modify the volfile to have an illegal string for the ratio
3. create split brain or under-quorum situation and check whether quorum is met

Actual results:

If the value is not parseable, then glusterd uses 0%
as a percentage, hence possibly leading to split brain.


Expected results:

If the value is not parseable as a percentage, then
glusterd should revert to the default of 50%+1 for quorum.

Comment 1 Worker Ant 2017-06-13 14:50:21 UTC

REVIEW: https://review.gluster.org/17538 (glusterd: fix quorum calculation if percentage is not specified correctly) posted (#1) for review on master by Michael Adam (obnox)

Comment 2 Michael Adam 2017-06-13 14:51:42 UTC

There's the patch.
It should be pretty obvious.
If a test is needed, I need to think about it.
Need to modify the volfile manually to create the situation, etc.

Comment 3 Worker Ant 2017-06-14 12:49:33 UTC

COMMIT: https://review.gluster.org/17538 committed in master by Atin Mukherjee (amukherj) 
------
commit c5b81783d5532a3b3a2bd1948bd1c9bfa4a023e1
Author: Michael Adam <obnox>
Date:   Tue Jun 13 10:06:57 2017 +0200

    glusterd: fix quorum calculation if percentage is not specified correctly
    
    There was an extra "ratio = _gf_true".
    
    - In case the ratio was specified correctly
      in the volfile, this is redundant.
    - In case the ratio was specified, but not parseable
      into a precentage, this is wrong and would
      lead to a quorum count of 0 instead of falling
      back to the default of 50% + 1.
    
    This patch removes the extra setting of "ratio".
    
    Change-Id: I2bd57ebf1b8989e905481a2b6285a1f422942f72
    BUG: 1461129
    Signed-off-by: Michael Adam <obnox>
    Reviewed-on: https://review.gluster.org/17538
    Reviewed-by: Atin Mukherjee <amukherj>
    Tested-by: Atin Mukherjee <amukherj>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>

Comment 4 Shyamsundar 2017-09-05 17:34:18 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.12.0, please open a new bug report.

glusterfs-3.12.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-September/000082.html
[2] https://www.gluster.org/pipermail/gluster-users/