Bug 1284453 - Dist-geo-rep: Support geo-replication to work with sharding
Dist-geo-rep: Support geo-replication to work with sharding
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: geo-replication (Show other bugs)
3.7.0
Unspecified Unspecified
high Severity unspecified
: ---
: ---
Assigned To: Kotresh HR
:
Depends On: 1265148
Blocks: 1275972
  Show dependency treegraph
 
Reported: 2015-11-23 06:41 EST by Kotresh HR
Modified: 2016-04-19 03:49 EDT (History)
2 users (show)

See Also:
Fixed In Version: glusterfs-3.7.7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1265148
Environment:
Last Closed: 2016-03-22 04:14:29 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Kotresh HR 2015-11-23 06:41:42 EST
+++ This bug was initially created as a clone of Bug #1265148 +++

Description of problem:
This bug tracks all the changes required for geo-replication to work with
sharding.

Requirements as per design:
  1. Both master and slave should be sharded
  2. Sharding configurations should be retained same as in master

Version-Release number of selected component (if applicable):
mainline

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:
Geo-replication does not work with sharding

Expected results:
Geo-replication should work with sharding.
Comment 1 Vijay Bellur 2015-11-23 06:52:43 EST
REVIEW: http://review.gluster.org/12721 (geo-rep: Handle FXATTROP and XATTROP) posted (#1) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)
Comment 2 Vijay Bellur 2015-11-24 00:56:49 EST
REVIEW: http://review.gluster.org/12729 (geo-rep: Fix syncing chown in xsync crawl) posted (#1) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)
Comment 3 Vijay Bellur 2015-11-24 01:28:06 EST
REVIEW: http://review.gluster.org/12731 (features/gfid-access: Fix entry creation via setxattr for geo-rep) posted (#1) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)
Comment 4 Vijay Bellur 2015-11-24 01:35:48 EST
REVIEW: http://review.gluster.org/12732 (features/changelog: Capture FXATTROP and XATTROP in changelog) posted (#1) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)
Comment 5 Vijay Bellur 2015-11-25 01:59:28 EST
REVIEW: http://review.gluster.org/12731 (features/gfid-access: Fix entry creation via setxattr for geo-rep) posted (#2) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)
Comment 6 Vijay Bellur 2015-11-25 02:22:52 EST
COMMIT: http://review.gluster.org/12732 committed in release-3.7 by Venky Shankar (vshankar@redhat.com) 
------
commit 9321f94f66143b1691cfa766dc202d8bd65dcd22
Author: Kotresh HR <khiremat@redhat.com>
Date:   Tue Sep 22 16:18:29 2015 +0530

    features/changelog: Capture FXATTROP and XATTROP in changelog
    
    GEO-REP INTEROP WITH SHARD FEATURE
    
    shard xlator updates size of the file using FXATTROP
    or XATTROP. Hence record the same in changelog.
    
    BUG: 1284453
    Change-Id: I2f14b6075f863c0bed3ee2a159085b9b536a756d
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12225
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
    Reviewed-on: http://review.gluster.org/12732
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
Comment 7 Vijay Bellur 2015-11-26 02:00:17 EST
REVIEW: http://review.gluster.org/12721 (geo-rep: Handle FXATTROP and XATTROP) posted (#2) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)
Comment 8 Vijay Bellur 2015-11-27 06:54:55 EST
COMMIT: http://review.gluster.org/12729 committed in release-3.7 by Venky Shankar (vshankar@redhat.com) 
------
commit 8b3061887319d38eb3644e5c4ec5a2b226de6723
Author: Kotresh HR <khiremat@redhat.com>
Date:   Mon Sep 21 14:51:13 2015 +0530

    geo-rep: Fix syncing chown in xsync crawl
    
    GEO-REP INTEROP WITH SHARD FEATURE
    
    Problem:
        The sequence of entry creation and chown in master
     is recorded as creation of entry with resulted
     user:group in xsync changelog. During sync, entry
     creation is always split into two ops, MKNOD and
     SETATTR. Hence the issue is not being hit otherwise
     it would have failed with EPERM if parent is owned
     by different user. But with shard translator being
     enabled on slave, doing entry creation with MKNOD and
     SETATTR is not allowed, SETATTR fails as it accesses
     inode structure which is not linked.
    
    Solution:
        The sequence of entry creation and chown in master
     should be recorded as MKNOD and SETATTR separately always
     and do entry creation with single op in gfid-access
     xlator. The gfid-access patch will be sent separately.
    
    BUG: 1284453
    Change-Id: Ia577aa5270cb96b86830885d6c4c01fb0133eeed
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12205
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
    Reviewed-on: http://review.gluster.org/12729
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Milind Changire <mchangir@redhat.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
Comment 9 Vijay Bellur 2015-11-27 06:55:43 EST
COMMIT: http://review.gluster.org/12721 committed in release-3.7 by Venky Shankar (vshankar@redhat.com) 
------
commit 6c451ec62dedc6fc4b739734eb5eb35d4ecfa6ce
Author: Kotresh HR <khiremat@redhat.com>
Date:   Thu Sep 24 12:59:08 2015 +0530

    geo-rep: Handle FXATTROP and XATTROP
    
    GEO-REP INTEROP WITH SHARD FEATURE
    
    If it is FXATTROP or XATTROP in changelog,
    add the gfid to rsync queue.
    
    BUG: 1284453
    Change-Id: If598da1543bbff7fe9228ba57c7e813ed2a3a7f2
    Reviewed-on: http://review.gluster.org/12226
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12721
    Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
    Reviewed-by: Milind Changire <mchangir@redhat.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
Comment 10 Vijay Bellur 2015-11-27 07:15:50 EST
COMMIT: http://review.gluster.org/12731 committed in release-3.7 by Venky Shankar (vshankar@redhat.com) 
------
commit 0f4bd0b927f90e70252e613c980bb196395db9ea
Author: Kotresh HR <khiremat@redhat.com>
Date:   Tue Sep 15 16:56:01 2015 +0530

    features/gfid-access: Fix entry creation via setxattr for geo-rep
    
    GEO-REP INTEROP WITH SHARD FEATURE
    
    Problem:
        Geo-replication uses setxattr interface of gfid-access
        xlator to create entries and send explicit setattr
        after entry creation to set uid and gid. But between
        entry creation and setattr, the inode would not be
        linked. Hence operation which accesses inode structure
        during setattr by any the below xlator fails.
    
    Solution:
        Linking inode would seem the obvious solution but,
        gfid-access xlator cannot link inodes and maintain
        it as it would result in same inode pointing
        to two different paths one being virtual .gfid/<gfid>
        path and other being actual path.
    
        The solution is to set uid and gid in frame->root->uid
        and frame->root->gid respectively from which posix
        extracts and sets.
    
    BUG: 1284453
    Change-Id: I881c3541f7b056f25ee25b382957d71c821113c1
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12206
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
    Reviewed-on: http://review.gluster.org/12731
    Reviewed-by: Milind Changire <mchangir@redhat.com>
Comment 11 Kaushal 2016-04-19 03:49:02 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.7, please open a new bug report.

glusterfs-3.7.7 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-users/2016-February/025292.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.