Bug 1111490

Summary: Dist-geo-rep : geo-rep xsync crawl takes too much time to sync meta data changes.
Product: [Community] GlusterFS Reporter: Aravinda VK <avishwan>
Component: geo-replicationAssignee: Aravinda VK <avishwan>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: mainlineCC: aavati, bugs, csaba, david.macdonald, gluster-bugs, nlevinki, nsathyan
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.6.0beta1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1111171 Environment:
Last Closed: 2014-11-11 08:35:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1111171    
Bug Blocks:    

Comment 1 Anand Avati 2014-06-20 07:43:28 UTC
REVIEW: http://review.gluster.org/8124 (geo-rep: Avoid duplicate stat in xsync changelog processing) posted (#1) for review on master by Aravinda VK (avishwan)

Comment 2 Anand Avati 2014-06-20 09:22:14 UTC
REVIEW: http://review.gluster.org/8124 (geo-rep: Avoid duplicate stat in xsync changelog processing) posted (#2) for review on master by Aravinda VK (avishwan)

Comment 3 Anand Avati 2014-06-26 05:51:25 UTC
REVIEW: http://review.gluster.org/8124 (geo-rep: Avoid duplicate stat in xsync changelog processing) posted (#3) for review on master by Aravinda VK (avishwan)

Comment 4 Anand Avati 2014-07-08 07:55:37 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#1) for review on master by Venky Shankar (vshankar)

Comment 5 Anand Avati 2014-07-09 08:52:31 UTC
COMMIT: http://review.gluster.org/8124 committed in master by Venky Shankar (vshankar) 
------
commit 56c7b07c746ba7ae86117f19bbf304e1150184fb
Author: Aravinda VK <avishwan>
Date:   Fri Jun 20 12:59:43 2014 +0530

    geo-rep: Avoid duplicate stat in xsync changelog processing
    
    When A file/dir is identified for metadata sync, it was doing
    duplicate stat to get the metadata to sync.
    
    With this patch it avoids doing one additional stat call. Xsync
    performance will improve.
    
    rsync will copy files metadata, so no need to include for
    processing.
    
    BUG: 1111490
    
    Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
    Signed-off-by: Aravinda VK <avishwan>
    Reviewed-on: http://review.gluster.org/8124
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>
    Tested-by: Venky Shankar <vshankar>

Comment 6 Anand Avati 2014-07-09 13:34:51 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#2) for review on master by Venky Shankar (vshankar)

Comment 7 Anand Avati 2014-07-09 17:05:04 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#3) for review on master by Venky Shankar (vshankar)

Comment 8 Anand Avati 2014-07-10 12:48:40 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#4) for review on master by Venky Shankar (vshankar)

Comment 9 Anand Avati 2014-07-10 12:48:44 UTC
REVIEW: http://review.gluster.org/8275 (test: utility to create files with a specific gfid) posted (#3) for review on master by Venky Shankar (vshankar)

Comment 10 Anand Avati 2014-07-11 06:29:42 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#5) for review on master by Venky Shankar (vshankar)

Comment 11 Anand Avati 2014-07-11 06:29:46 UTC
REVIEW: http://review.gluster.org/8275 (test: utility to create files with a specific gfid) posted (#4) for review on master by Venky Shankar (vshankar)

Comment 12 Anand Avati 2014-07-14 13:27:01 UTC
COMMIT: http://review.gluster.org/8260 committed in master by Vijay Bellur (vbellur) 
------
commit eeab758c7c7e7670f86fc1d8c3785a1ecb7208b4
Author: Venky Shankar <vshankar>
Date:   Tue Jul 8 13:05:15 2014 +0530

    features/gfid-access: error handling for entry creation
    
    Proceed with setattr() only on a successfull entry creation.
    Winding a setattr() using a freshlyOC initiated inode would
    most likely fail in one translator or the other (e.g. DHT
    expecting the layout information to be set in the inode
    context), which is the case if the inode was not looked up.
    
    Therefore, gfid-access handles failure entry creations and
    passes the _correct_ errno back to the client instead of
    continuing with setattr() call and probably returning back
    incorrect errno. Also, filling up inode->gfid is required
    as the new inode is not looked up and ->gfid would be
    certainely required for inode operations.
    
    Change-Id: Ie92f5647a89bf558c07710ab0400bce69d59fc31
    BUG: 1111490
    Signed-off-by: Venky Shankar <vshankar>
    Reviewed-on: http://review.gluster.org/8260
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kotresh HR <khiremat>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 13 Anand Avati 2014-07-14 13:42:56 UTC
COMMIT: http://review.gluster.org/8275 committed in master by Vijay Bellur (vbellur) 
------
commit 23ae9f3ad79d2d83447096d8b3627491b2e74bb8
Author: Venky Shankar <vshankar>
Date:   Wed Jul 9 18:36:18 2014 +0530

    test: utility to create files with a specific gfid
    
    python test utility that uses setxattr() interface to
    create files with a predefined gfid (which is also used
    by geo-replication). this utility aids in writing test
    cases for gfid-access translator.
    
    Change-Id: Ib11248844a094cc02dae74d969f2e9212f94ede4
    BUG: 1111490
    Signed-off-by: Venky Shankar <vshankar>
    Reviewed-on: http://review.gluster.org/8275
    Reviewed-by: Aravinda VK <avishwan>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 14 Niels de Vos 2014-09-22 12:43:22 UTC
A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 15 Niels de Vos 2014-11-11 08:35:33 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users