Bug 1187140 - [RFE]: geo-rep: Tool to find missing files in slave volume
Summary: [RFE]: geo-rep: Tool to find missing files in slave volume
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: geo-replication
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kotresh HR
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-01-29 12:09 UTC by Kotresh HR
Modified: 2015-05-14 17:35 UTC (History)
4 users (show)

Fixed In Version: glusterfs-3.7.0beta1
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-05-14 17:26:25 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Kotresh HR 2015-01-29 12:09:32 UTC
Description of problem:

Need a tool to find missing files in slave volume.
Now if geo-rep misses a file to sync, whole of gfids from changelog
is dumped. There is no way we can know the exact files that missed
to sync.

Comment 1 Anand Avati 2015-01-29 12:10:11 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#1) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 2 Anand Avati 2015-02-04 12:01:10 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#2) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 3 Anand Avati 2015-02-05 08:01:05 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#3) for review on master by Aravinda VK (avishwan@redhat.com)

Comment 4 Anand Avati 2015-02-05 09:38:27 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#4) for review on master by Aravinda VK (avishwan@redhat.com)

Comment 5 Anand Avati 2015-02-05 13:24:05 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#5) for review on master by Aravinda VK (avishwan@redhat.com)

Comment 6 Anand Avati 2015-02-05 13:53:34 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#6) for review on master by Aravinda VK (avishwan@redhat.com)

Comment 7 Anand Avati 2015-03-06 03:08:02 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#7) for review on master by Aravinda VK (avishwan@redhat.com)

Comment 8 Anand Avati 2015-03-11 10:12:18 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#8) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 9 Anand Avati 2015-03-12 10:57:45 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#9) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 10 Anand Avati 2015-03-13 04:10:10 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#10) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 11 Anand Avati 2015-03-13 05:35:11 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#11) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 12 Anand Avati 2015-03-13 05:48:18 UTC
REVIEW: http://review.gluster.org/9503 (tools: Finds missing files in gluster volume given backend brickpath) posted (#12) for review on master by Kotresh HR (khiremat@redhat.com)

Comment 13 Anand Avati 2015-03-16 04:20:23 UTC
COMMIT: http://review.gluster.org/9503 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit 7a9a66cc5fb7f06118fab1fc2ae1c43cfbb1178f
Author: Kotresh HR <khiremat@redhat.com>
Date:   Thu Jan 29 15:53:19 2015 +0530

    tools: Finds missing files in gluster volume given backend brickpath
    
    The tool finds the missing files in a geo-replication slave volume.
    The tool crawls backend .glusterfs of the brickpath, which is passed
    as a parameter and stats each entry on slave volume mount to check
    the presence of file. The mount used is aux-gfid-mount, hence no path
    conversion is required and is fast. The tool needs to be run on every
    node in cluster for each brickpath of geo-rep master volume to find
    missing files on slave volume. The tool is generic enough and can be
    used in non geo-replication context as well.
    
    Most of the crawler code is leverged from Avati's xfind and is modified
    to crawl only .glusterfs (https://github.com/avati/xsync)
    
    Thanks Aravinda for scripts to convert gfid to path.
    
    Change-Id: I84deaaaf638f7c571ff1319b67a3440fe27da810
    BUG: 1187140
    Signed-off-by: Aravinda VK <avishwan@redhat.com>
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/9503
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>

Comment 14 Niels de Vos 2015-05-14 17:26:25 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 15 Niels de Vos 2015-05-14 17:28:20 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 16 Niels de Vos 2015-05-14 17:35:15 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.