Bug 1104653 - DHT + rebalance : rebalance process crashed + data loss + few Directories are present on sub-volumes but not visible on mount point + lookup is not healing directories
Summary: DHT + rebalance : rebalance process crashed + data loss + few Directories are...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: distribute
Version: pre-release
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
Assignee: vsomyaju
QA Contact:
URL:
Whiteboard:
Depends On: 1034108
Blocks: 1098971 1138922 1139984
TreeView+ depends on / blocked
 
Reported: 2014-06-04 12:43 UTC by vsomyaju
Modified: 2015-11-30 17:04 UTC (History)
8 users (show)

Fixed In Version: glusterfs-3.6.0beta1
Doc Type: Bug Fix
Doc Text:
Clone Of: 1034108
: 1138383 1138922 1139984 (view as bug list)
Environment:
Last Closed: 2014-11-11 08:34:07 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Comment 1 Anand Avati 2014-06-04 12:48:31 UTC
REVIEW: http://review.gluster.org/7973 (dht/rebalance: Do not allow rebalance when gfid mismatch found) posted (#1) for review on master by venkatesh somyajulu (vsomyaju)

Comment 2 Anand Avati 2014-06-05 11:21:19 UTC
REVIEW: http://review.gluster.org/7992 (dht/rebalance:  Count as failure if readdir provides an entry on which lookup fails) posted (#1) for review on master by venkatesh somyajulu (vsomyaju)

Comment 3 Anand Avati 2014-06-05 11:22:46 UTC
REVIEW: http://review.gluster.org/7992 (dht/rebalance: Count it as failure if lookup on readdirp entry fail) posted (#2) for review on master by venkatesh somyajulu (vsomyaju)

Comment 4 Anand Avati 2014-06-06 05:09:48 UTC
REVIEW: http://review.gluster.org/7973 (dht/rebalance: Do not allow rebalance when gfid mismatch found) posted (#2) for review on master by venkatesh somyajulu (vsomyaju)

Comment 5 Anand Avati 2014-06-12 08:51:40 UTC
REVIEW: http://review.gluster.org/7973 (dht/rebalance: Do not allow rebalance when gfid mismatch found) posted (#3) for review on master by venkatesh somyajulu (vsomyaju)

Comment 6 Anand Avati 2014-06-12 12:43:56 UTC
REVIEW: http://review.gluster.org/7973 (dht/rebalance: Do not allow rebalance when gfid mismatch found) posted (#4) for review on master by venkatesh somyajulu (vsomyaju)

Comment 7 Anand Avati 2014-06-13 17:30:16 UTC
REVIEW: http://review.gluster.org/7973 (dht/rebalance: Do not allow rebalance when gfid mismatch found) posted (#5) for review on master by Raghavendra G (rgowdapp)

Comment 8 Anand Avati 2014-06-14 17:55:56 UTC
COMMIT: http://review.gluster.org/7973 committed in master by Vijay Bellur (vbellur) 
------
commit 72c7afcd091605e06a02e7c5de0eff18f0690a82
Author: Venkatesh Somyajulu <vsomyaju>
Date:   Thu Jun 12 14:18:05 2014 +0530

    dht/rebalance: Do not allow rebalance when gfid mismatch found
    
    Due to race condition, it may so happen that, gfid obtained
    in readdirp and gfid found by lookup are different for a given
    name. in that case do no allow the rebalance.
    
    Readdirp of an entry will bring the gfid, which will be stored
    in the inode through inode_link, and when lookup is done and
    gfid brought by lookup is different from the one stored in the
    inode, client3_3_lookup_cbk will return ESATLE and error will be
    captured by rebalance process.
    
    Change-Id: Iad839177ef9b80c1dd0e87f3406bcf4cb018e6fa
    BUG: 1104653
    Signed-off-by: Venkatesh Somyajulu <vsomyaju>
    Reviewed-on: http://review.gluster.org/7973
    Reviewed-by: Shyamsundar Ranganathan <srangana>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Krutika Dhananjay <kdhananj>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 9 Anand Avati 2014-06-16 10:12:33 UTC
REVIEW: http://review.gluster.org/8073 (cluster/dht:  make loc->inode instead of loc->parent) posted (#1) for review on master by venkatesh somyajulu (vsomyaju)

Comment 10 Anand Avati 2014-06-16 16:11:40 UTC
COMMIT: http://review.gluster.org/8073 committed in master by Vijay Bellur (vbellur) 
------
commit de22a20a76f5cf0af3ca2898f0bfd8324a191e0c
Author: Venkatesh Somyajulu <vsomyaju>
Date:   Mon Jun 16 15:39:57 2014 +0530

    cluster/dht:  make loc->inode instead of loc->parent
    
    parent's inode should be taken from loc->inode.
    
    Change-Id: I979b7333efa93b1e8f4c73ccf048d48e308f9289
    BUG: 1104653
    Signed-off-by: Venkatesh Somyajulu <vsomyaju>
    Reviewed-on: http://review.gluster.org/8073
    Reviewed-by: Krutika Dhananjay <kdhananj>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 11 Niels de Vos 2014-09-22 12:41:53 UTC
A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 12 Niels de Vos 2014-11-11 08:34:07 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users


Note You need to log in before you can comment on or make changes to this bug.