Bug 1111490 - Dist-geo-rep : geo-rep xsync crawl takes too much time to sync meta data changes.
Summary: Dist-geo-rep : geo-rep xsync crawl takes too much time to sync meta data chan...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: geo-replication
Version: mainline
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Aravinda VK
QA Contact:
URL:
Whiteboard:
Depends On: 1111171
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-06-20 07:34 UTC by Aravinda VK
Modified: 2014-11-11 08:35 UTC (History)
7 users (show)

Fixed In Version: glusterfs-3.6.0beta1
Doc Type: Bug Fix
Doc Text:
Clone Of: 1111171
Environment:
Last Closed: 2014-11-11 08:35:33 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Comment 1 Anand Avati 2014-06-20 07:43:28 UTC
REVIEW: http://review.gluster.org/8124 (geo-rep: Avoid duplicate stat in xsync changelog processing) posted (#1) for review on master by Aravinda VK (avishwan)

Comment 2 Anand Avati 2014-06-20 09:22:14 UTC
REVIEW: http://review.gluster.org/8124 (geo-rep: Avoid duplicate stat in xsync changelog processing) posted (#2) for review on master by Aravinda VK (avishwan)

Comment 3 Anand Avati 2014-06-26 05:51:25 UTC
REVIEW: http://review.gluster.org/8124 (geo-rep: Avoid duplicate stat in xsync changelog processing) posted (#3) for review on master by Aravinda VK (avishwan)

Comment 4 Anand Avati 2014-07-08 07:55:37 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#1) for review on master by Venky Shankar (vshankar)

Comment 5 Anand Avati 2014-07-09 08:52:31 UTC
COMMIT: http://review.gluster.org/8124 committed in master by Venky Shankar (vshankar) 
------
commit 56c7b07c746ba7ae86117f19bbf304e1150184fb
Author: Aravinda VK <avishwan>
Date:   Fri Jun 20 12:59:43 2014 +0530

    geo-rep: Avoid duplicate stat in xsync changelog processing
    
    When A file/dir is identified for metadata sync, it was doing
    duplicate stat to get the metadata to sync.
    
    With this patch it avoids doing one additional stat call. Xsync
    performance will improve.
    
    rsync will copy files metadata, so no need to include for
    processing.
    
    BUG: 1111490
    
    Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
    Signed-off-by: Aravinda VK <avishwan>
    Reviewed-on: http://review.gluster.org/8124
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>
    Tested-by: Venky Shankar <vshankar>

Comment 6 Anand Avati 2014-07-09 13:34:51 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#2) for review on master by Venky Shankar (vshankar)

Comment 7 Anand Avati 2014-07-09 17:05:04 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#3) for review on master by Venky Shankar (vshankar)

Comment 8 Anand Avati 2014-07-10 12:48:40 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#4) for review on master by Venky Shankar (vshankar)

Comment 9 Anand Avati 2014-07-10 12:48:44 UTC
REVIEW: http://review.gluster.org/8275 (test: utility to create files with a specific gfid) posted (#3) for review on master by Venky Shankar (vshankar)

Comment 10 Anand Avati 2014-07-11 06:29:42 UTC
REVIEW: http://review.gluster.org/8260 (features/gfid-access: error handling for entry creation) posted (#5) for review on master by Venky Shankar (vshankar)

Comment 11 Anand Avati 2014-07-11 06:29:46 UTC
REVIEW: http://review.gluster.org/8275 (test: utility to create files with a specific gfid) posted (#4) for review on master by Venky Shankar (vshankar)

Comment 12 Anand Avati 2014-07-14 13:27:01 UTC
COMMIT: http://review.gluster.org/8260 committed in master by Vijay Bellur (vbellur) 
------
commit eeab758c7c7e7670f86fc1d8c3785a1ecb7208b4
Author: Venky Shankar <vshankar>
Date:   Tue Jul 8 13:05:15 2014 +0530

    features/gfid-access: error handling for entry creation
    
    Proceed with setattr() only on a successfull entry creation.
    Winding a setattr() using a freshlyOC initiated inode would
    most likely fail in one translator or the other (e.g. DHT
    expecting the layout information to be set in the inode
    context), which is the case if the inode was not looked up.
    
    Therefore, gfid-access handles failure entry creations and
    passes the _correct_ errno back to the client instead of
    continuing with setattr() call and probably returning back
    incorrect errno. Also, filling up inode->gfid is required
    as the new inode is not looked up and ->gfid would be
    certainely required for inode operations.
    
    Change-Id: Ie92f5647a89bf558c07710ab0400bce69d59fc31
    BUG: 1111490
    Signed-off-by: Venky Shankar <vshankar>
    Reviewed-on: http://review.gluster.org/8260
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kotresh HR <khiremat>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 13 Anand Avati 2014-07-14 13:42:56 UTC
COMMIT: http://review.gluster.org/8275 committed in master by Vijay Bellur (vbellur) 
------
commit 23ae9f3ad79d2d83447096d8b3627491b2e74bb8
Author: Venky Shankar <vshankar>
Date:   Wed Jul 9 18:36:18 2014 +0530

    test: utility to create files with a specific gfid
    
    python test utility that uses setxattr() interface to
    create files with a predefined gfid (which is also used
    by geo-replication). this utility aids in writing test
    cases for gfid-access translator.
    
    Change-Id: Ib11248844a094cc02dae74d969f2e9212f94ede4
    BUG: 1111490
    Signed-off-by: Venky Shankar <vshankar>
    Reviewed-on: http://review.gluster.org/8275
    Reviewed-by: Aravinda VK <avishwan>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 14 Niels de Vos 2014-09-22 12:43:22 UTC
A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 15 Niels de Vos 2014-11-11 08:35:33 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users


Note You need to log in before you can comment on or make changes to this bug.