Bug 1361101 - [RFE] arbiter for 3 way replication
Summary: [RFE] arbiter for 3 way replication
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: replicate
Version: rhgs-3.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.2.0
Assignee: Ravishankar N
QA Contact: Karan Sandha
URL:
Whiteboard:
Depends On: 1199985
Blocks: 1217689 Gluster-HC-2 1351503
TreeView+ depends on / blocked
 
Reported: 2016-07-28 10:57 UTC by Pranith Kumar K
Modified: 2017-03-23 05:41 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.8.4-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1199985
Environment:
Last Closed: 2017-03-23 05:41:37 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0486 normal SHIPPED_LIVE Moderate: Red Hat Gluster Storage 3.2.0 security, bug fix, and enhancement update 2017-03-23 09:18:45 UTC
Red Hat Bugzilla 1365850 None None None Never

Internal Links: 1365850

Description Pranith Kumar K 2016-07-28 10:57:05 UTC
+++ This bug was initially created as a clone of Bug #1199985 +++

Provide a mechanism in a 3 way replica to avoid split-brains.
See http://review.gluster.org/#/c/9656/

--- Additional comment from Anand Avati on 2015-03-09 11:53:24 EDT ---

REVIEW: http://review.gluster.org/9840 (afr: arbiter xlator) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-03-16 07:37:22 EDT ---

REVIEW: http://review.gluster.org/9840 (afr: arbiter xlator) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-03-18 00:22:15 EDT ---

REVIEW: http://review.gluster.org/9840 (afr: arbiter xlator) posted (#3) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-03-18 02:53:23 EDT ---

REVIEW: http://review.gluster.org/9840 (afr: arbiter xlator) posted (#4) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-03-18 06:34:43 EDT ---

REVIEW: http://review.gluster.org/9840 (afr: arbiter xlator) posted (#5) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-03-19 08:36:57 EDT ---

REVIEW: http://review.gluster.org/9840 (afr: arbiter xlator) posted (#6) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-03-19 13:45:54 EDT ---

COMMIT: http://review.gluster.org/9840 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit 32ed7aa5ad7049a9d85c795f997336c0366151a8
Author: Ravishankar N <ravishankar@redhat.com>
Date:   Thu Mar 19 12:33:51 2015 +0000

    afr: arbiter xlator
    
    This patch adds the arbiter translator into the tree. This is a server
    side xlator used for replica 3 volumes. It sits above posix and will be
    loaded on the 3rd (last) brick of every afr subvolume in a replica 3
    configuration. It intercepts inode read/write operations: reads are
    unwound with ENOTCONN, inode writes are unwound with success without
    actually passing them down to posix. Metadata operations are allowed to
    pass through.
    
    The CLI for creating a 3 way replica with arbiter is also added but kept
    disabled (A 'normal' 3 way replica is created instead).
    
    This patch is a part of the arbiter logic implementation for 3 way AFR,
    details of which can be found at http://review.gluster.org/#/c/9656/
    
    Change-Id: I395b81f49d5da52c466daf5c8518f1bbad9c16fa
    BUG: 1199985
    Signed-off-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-on: http://review.gluster.org/9840
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>

--- Additional comment from Anand Avati on 2015-03-19 15:33:50 EDT ---

REVIEW: http://review.gluster.org/9950 (build: remove broken and unused _la_LIBADD variable) posted (#1) for review on master by Niels de Vos (ndevos@redhat.com)

--- Additional comment from Anand Avati on 2015-03-19 23:55:20 EDT ---

COMMIT: http://review.gluster.org/9950 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit 7fea544079c0fdc2d1d6a29e039cc925f81b2117
Author: Niels de Vos <ndevos@redhat.com>
Date:   Thu Mar 19 20:30:13 2015 +0100

    build: remove broken and unused _la_LIBADD variable
    
    While building, the following warning is displayed:
    
      xlators/features/arbiter/src/Makefile.am:7: variable `_la_LIBADD' is defined but no program or
      xlators/features/arbiter/src/Makefile.am:7: library has `_la' as canonical name (possible typo)
    
    The _la_LIBADD really seems like a typo, dropping it should not be harmful to
    anything, except for the warning that will now be gone.
    
    BUG: 1199985
    Change-Id: I3f3ba911f59df2e51fdc6387295fff4bbcc5a12d
    Signed-off-by: Niels de Vos <ndevos@redhat.com>
    Reviewed-on: http://review.gluster.org/9950
    Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>

--- Additional comment from Ravishankar N on 2015-03-30 21:09:30 EDT ---

I'm keeping the BZ in post state (and not modified) as I plan to submit further patches for this feature against this BZ until 3.7 is released.

--- Additional comment from Anand Avati on 2015-04-15 13:50:28 EDT ---

REVIEW: http://review.gluster.org/10257 (arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-15 13:50:31 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-17 07:55:21 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-17 07:55:26 EDT ---

REVIEW: http://review.gluster.org/10257 (arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-23 04:20:31 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#3) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-23 04:20:34 EDT ---

REVIEW: http://review.gluster.org/10257 (arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol) posted (#3) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-24 01:24:46 EDT ---

REVIEW: http://review.gluster.org/10257 (arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol) posted (#4) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-24 12:12:52 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#4) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-24 12:12:54 EDT ---

REVIEW: http://review.gluster.org/10257 (arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol) posted (#5) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-27 10:39:16 EDT ---

REVIEW: http://review.gluster.org/10257 (arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol) posted (#6) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-28 01:40:33 EDT ---

COMMIT: http://review.gluster.org/10257 committed in master by Krishnan Parthasarathi (kparthas@redhat.com) 
------
commit d4889b2cfd29e6ecc911d2b29d1f85d516a66eaf
Author: Ravishankar N <ravishankar@redhat.com>
Date:   Wed Apr 15 22:53:21 2015 +0530

    arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvol
    
    Logic for adding the 'glusterd_brickinfo->group' member and using it to
    find the brick positon has been taken from http://review.gluster.org/#/c/9919.
    Thanks to Jeff Darcy for that.
    
    This patch is a part of the arbiter logic implementation for 3 way AFR
    details of which can be found at http://review.gluster.org/#/c/9656/
    
    Change-Id: Idbfe4f29ee8e098e0102def8f38b32314316b188
    BUG: 1199985
    Signed-off-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-on: http://review.gluster.org/10257
    Tested-by: NetBSD Build System
    Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
    Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>

--- Additional comment from Anand Avati on 2015-04-30 04:43:16 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#5) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-04-30 22:52:47 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#6) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-05-01 03:11:05 EDT ---

REVIEW: http://review.gluster.org/10258 (afr: add arbitration support) posted (#7) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-05-05 00:40:55 EDT ---

REVIEW: http://review.gluster.org/10541 (doc: AFR arbiter volume usage) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Anand Avati on 2015-05-05 03:06:55 EDT ---

COMMIT: http://review.gluster.org/10258 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit 25e8e74eb7b81ccd114a9833371a3f72d284c48d
Author: Ravishankar N <ravishankar@redhat.com>
Date:   Wed Apr 15 22:22:08 2015 +0530

    afr: add arbitration support
    
    Add logic in afr to work in conjunction with the arbiter xlator when a
    replica 3 arbiter volume is created. More specifically, this patch:
    
    * Enables full locks for afr data transaction for such volumes.
    * Removes the upfront marking of pending xattrs at the time of pre-op
      and defer it to post-op. (This is an arbiter independent change and is made for all afr transactions.)
    * After pre-op stage, check if we can proceed with the fop stage without
      ending up in split-brain by examining the changelog xattrs.
    * Unwinds the fop with failure if only one source was available at the
       time of pre-op and the fop happened to fail on particular source brick.
    * Skips data self-heal if arbiter brick is the only source available.
    * Adds the arbiter-count option to the shd graph.
    
    This patch is a part of the arbiter logic implementation for 3 way AFR
    details of which can be found at http://review.gluster.org/#/c/9656/
    
    Change-Id: I9603db9d04de5626eb2f4d8d959ef5b46113561d
    BUG: 1199985
    Signed-off-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-on: http://review.gluster.org/10258
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

--- Additional comment from Anand Avati on 2015-05-05 05:14:45 EDT ---

REVIEW: http://review.gluster.org/10541 (doc: AFR arbiter volume usage) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)

--- Additional comment from Nagaprasad Sathyanarayana on 2015-10-25 11:02:22 EDT ---

Fix for this BZ is already present in a GlusterFS release. You can find clone of this BZ, fixed in a GlusterFS release and closed. Hence closing this mainline BZ as well.

--- Additional comment from Niels de Vos on 2016-06-16 08:42:14 EDT ---

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 9 errata-xmlrpc 2017-03-23 05:41:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0486.html


Note You need to log in before you can comment on or make changes to this bug.