Bug 1292084 - [georep+tiering]: Geo-replication sync is broken if cold tier is EC
[georep+tiering]: Geo-replication sync is broken if cold tier is EC
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: tiering (Show other bugs)
mainline
x86_64 Linux
urgent Severity urgent
: ---
: ---
Assigned To: Satish Mohan
bugs@gluster.org
: ZStream
Depends On: 1291195
Blocks: 1293309
  Show dependency treegraph
 
Reported: 2015-12-16 07:40 EST by Gaurav Kumar Garg
Modified: 2016-06-16 09:50 EDT (History)
12 users (show)

See Also:
Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1291195
: 1293309 (view as bug list)
Environment:
Last Closed: 2016-06-16 09:50:45 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 1 Vijay Bellur 2015-12-16 07:44:04 EST
REVIEW: http://review.gluster.org/12982 (cli/xml: display correct xml output of tier volume) posted (#1) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)
Comment 2 Kotresh HR 2015-12-17 07:29:14 EST
Description of problem:
=======================

If cold tier is Distributed-Disperse (2x{4+2}) and hot tier is distributed-replicate(2x2), then the total subvolumes in the system are 4. But the lock files created under shared storage are 3 and hence only 3 bricks from 3 subvolume acquires the lock and participate in syncing. While the remaining one subvolume never participates in syncing. 

But if both cold and hot tier are Distributed-Replicate (2x2), then the lock files created are 4 and all 4 subvolume participates in syncing

I am suspecting an issue with the xml output generation of a volume file. 

A) If cold tier is Distributed-Disperse and hot tier is Distributed-Replicate, the xml output wrongly shows hot tier as REPLICATE: Example:

            <hotBrickType>Replicate</hotBrickType>
            <numberOfBricks>0 x 6 = 4</numberOfBricks>

B) If cold tier and hot tier both are Distributed-Replicate, the xml output is correctly shows hot tier as Distributed-Replicate: Example:

            <hotBrickType>Distributed-Replicate</hotBrickType>
            <numberOfBricks>2 x 2 = 4</numberOfBricks>

Version-Release number of selected component (if applicable):
=============================================================
mainline


How reproducible:
=================
2/2

Steps to Reproduce:
===================
1. Create master and slave cluster
2. Create Master volume (Cold Tier as distributed-disperse and Hot tier as Distributed-Replicate)
3. Create Slave volume (Distributed-Replicate)
4. Create and Start geo-rep session between master and slave

Actual results:
===============

Only brick from one subvolume in hot tier becomes ACTIVE


Expected results:
================

One brick from each subvolume in hot tier should become ACTIVE
Comment 3 Vijay Bellur 2015-12-17 07:32:29 EST
REVIEW: http://review.gluster.org/12994 (geo-rep: Fix getting subvol number) posted (#1) for review on master by Kotresh HR (khiremat@redhat.com)
Comment 4 Kotresh HR 2015-12-17 07:34:37 EST
Two patches one from cli xml and other from geo-rep is needed to fix this issue.

1. cli xml: http://review.gluster.org/12982
2. Geo-rep: http://review.gluster.org/12994
Comment 5 Vijay Bellur 2015-12-17 07:34:54 EST
REVIEW: http://review.gluster.org/12982 (cli/xml: display correct xml output of tier volume) posted (#2) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)
Comment 6 Vijay Bellur 2015-12-17 08:41:30 EST
REVIEW: http://review.gluster.org/12994 (geo-rep: Fix getting subvol number) posted (#2) for review on master by Kotresh HR (khiremat@redhat.com)
Comment 7 Vijay Bellur 2015-12-18 03:54:59 EST
REVIEW: http://review.gluster.org/12994 (geo-rep: Fix getting subvol number) posted (#3) for review on master by Kotresh HR (khiremat@redhat.com)
Comment 8 Vijay Bellur 2015-12-21 01:58:49 EST
REVIEW: http://review.gluster.org/12982 (cli/xml: display correct xml output of tier volume) posted (#3) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)
Comment 9 Vijay Bellur 2015-12-21 03:28:32 EST
COMMIT: http://review.gluster.org/12994 committed in master by Venky Shankar (vshankar@redhat.com) 
------
commit d677e195cb85bef28fcd9e2f45e487c9ea792311
Author: Kotresh HR <khiremat@redhat.com>
Date:   Thu Dec 17 12:39:30 2015 +0530

    geo-rep: Fix getting subvol number
    
    Fix getting subvol number if the volume
    type is tier. If the volume type was tier,
    the subvol number was calculated incorrectly
    and hence few of workers didn't become ACTIVE
    resulting in files not being replicated from
    corresponding brick. This patch addresses
    the same.
    
    Change-Id: Ic10ad7f09a0fa91b4bf2aa361dea3bd48be74853
    BUG: 1292084
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12994
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
Comment 10 Vijay Bellur 2015-12-21 06:46:10 EST
COMMIT: http://review.gluster.org/12982 committed in master by Atin Mukherjee (amukherj@redhat.com) 
------
commit b0e126d0edf10946701c2fd4f0f1cf8c7b07eda1
Author: Gaurav Kumar Garg <garg.gaurav52@gmail.com>
Date:   Wed Dec 16 18:04:55 2015 +0530

    cli/xml: display correct xml output of tier volume
    
    Currently When hot tier type is distributed-replicate and cold tier
    type is disperse volume then #gluster volume info --xml command is
    not giving its correct output. In case of HOT tier case its displaying
    wrong volume type.
    
    With this fix it will show correct xml output for tier volume
    irrespective of all the type of the volume's.
    
    Change-Id: If1de8d52d1e0ef3d0523163abed37b2b571715e8
    BUG: 1292084
    Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
    Reviewed-on: http://review.gluster.org/12982
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: mohammed rafi  kc <rkavunga@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Comment 11 Vijay Bellur 2015-12-22 02:06:33 EST
REVIEW: http://review.gluster.org/13062 (geo-rep: Fix getting subvol count) posted (#1) for review on master by Kotresh HR (khiremat@redhat.com)
Comment 12 Vijay Bellur 2015-12-22 10:06:49 EST
COMMIT: http://review.gluster.org/13062 committed in master by Venky Shankar (vshankar@redhat.com) 
------
commit 074158e7081ff0118c719aac7cf1bcde92ee8f7d
Author: Kotresh HR <khiremat@redhat.com>
Date:   Tue Dec 22 12:29:32 2015 +0530

    geo-rep: Fix getting subvol count
    
    Tiering doesn't support disperse volume as hot tier,
    hence xml output doesn't give 'hotdisperseCount'.
    Remove the usage of 'hotdisperseCount' in geo-rep
    and return 0 instead.
    
    Change-Id: I736e29257de085a25e38eb02959caad3465ebcda
    BUG: 1292084
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/13062
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Vivek
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
Comment 15 Niels de Vos 2016-06-16 09:50:45 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.