Bug 907072

Summary:	concurrent mkdir results in GFID mismatch
Product:	[Community] GlusterFS	Reporter:	Anand Avati <aavati>
Component:	distribute	Assignee:	shishir gowda <sgowda>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	urgent
Version:	mainline	CC:	amarts, chrisw, gluster-bugs, nsathyan, pportant
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.4.0	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	915992 (view as bug list)		Environment:
Last Closed:	2013-07-24 17:56:43 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	903396, 915992, 978061

Description Anand Avati 2013-02-03 03:18:12 UTC

when multiple clients attempt creation of same nested directories (e.g UFO object servers trying to create the implicit directories in an object name having "/" characters in them) results in some of those directories to get mismatched GFID, further resulting in leaked locks when replicate is also loaded.

Comment 1 Anand Avati 2013-02-03 03:34:16 UTC

*** Bug 906966 has been marked as a duplicate of this bug. ***

Comment 2 Vijay Bellur 2013-02-03 20:14:32 UTC

CHANGE: http://review.gluster.org/4459 (cluster/dht: ignore EEXIST error in mkdir to avoid GFID mismatch) merged in master by Anand Avati (avati)

Comment 3 Anand Avati 2013-09-07 04:33:17 UTC

REVIEW: http://review.gluster.org/5849 (cluster/dht: assign layout onto missing directories too) posted (#1) for review on master by Anand Avati (avati)

Comment 4 Anand Avati 2013-09-07 19:37:55 UTC

REVIEW: http://review.gluster.org/5849 (cluster/dht: assign layout onto missing directories too) posted (#2) for review on master by Anand Avati (avati)

Comment 5 Anand Avati 2013-09-07 19:44:32 UTC

REVIEW: http://review.gluster.org/5849 (cluster/dht: assign layout onto missing directories too) posted (#3) for review on master by Anand Avati (avati)

Comment 6 Anand Avati 2013-09-09 21:58:52 UTC

COMMIT: http://review.gluster.org/5849 committed in master by Anand Avati (avati) 
------
commit ebcf1c8ddb76ca1234282e5189f6800d89db4b98
Author: Anand Avati <avati>
Date:   Thu Aug 29 23:35:23 2013 -0700

    cluster/dht: assign layout onto missing directories too
    
    The current self-healing algorithm is ignoring missing directories
    for assigning new layout. When lookup() is racing against mkdir()
    or when self-healing a half-done mkdir(), the layout assignment split
    must happen based on the final number of directories, and not the
    currently existing number of directories (because we finish mkdir()
    of missing directories before hash layout assignment).
    
    Without this fix, concurrent mkdir() and lookup() will step on
    each others feet, create a messed up layout on disk, and end up
    with different in-memory layouts.
    
    Once two clients have different in-memory layouts, creation of
    subdirectory will not arbitrate on the same hashed subvolume and will
    result in GFID mismatch of the sub-directory.
    
    Change-Id: Ia47acad67c265060405984c822b4d37512b9dbb3
    BUG: 907072
    Signed-off-by: Anand Avati <avati>
    Reviewed-on: http://review.gluster.org/5849
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Amar Tumballi <amarts>
    Reviewed-by: Peter Portante <pportant>
    Tested-by: Peter Portante <pportant>

Comment 7 Anand Avati 2013-09-10 00:50:13 UTC

REVIEW: http://review.gluster.org/5871 (cluster/dht: assign layout onto missing directories too) posted (#1) for review on release-3.4 by Anand Avati (avati)

Comment 8 Anand Avati 2013-09-10 01:04:11 UTC

REVIEW: http://review.gluster.org/5871 (cluster/dht: assign layout onto missing directories too) posted (#2) for review on release-3.4 by Anand Avati (avati)

Comment 9 Anand Avati 2013-09-10 08:15:34 UTC

COMMIT: http://review.gluster.org/5871 committed in release-3.4 by Vijay Bellur (vbellur) 
------
commit 23f0ad993c5b12e8a8444fa2519864ccfc9cd8cf
Author: Anand Avati <avati>
Date:   Thu Aug 29 23:35:23 2013 -0700

    cluster/dht: assign layout onto missing directories too
    
    The current self-healing algorithm is ignoring missing directories
    for assigning new layout. When lookup() is racing against mkdir()
    or when self-healing a half-done mkdir(), the layout assignment split
    must happen based on the final number of directories, and not the
    currently existing number of directories (because we finish mkdir()
    of missing directories before hash layout assignment).
    
    Without this fix, concurrent mkdir() and lookup() will step on
    each others feet, create a messed up layout on disk, and end up
    with different in-memory layouts.
    
    Once two clients have different in-memory layouts, creation of
    subdirectory will not arbitrate on the same hashed subvolume and will
    result in GFID mismatch of the sub-directory.
    
    Change-Id: Ia47acad67c265060405984c822b4d37512b9dbb3
    BUG: 907072
    Signed-off-by: Anand Avati <avati>
    Reviewed-on: http://review.gluster.org/5871
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>