These have been making people in IRC scratch their heads for some times, but I just happened to find a reproducible case on my own systems today. The problem is that glusterd_is_brickpath_available is continuing to use the output string from realpath(3) even when that failed. The man page has this to say: Otherwise it returns a NULL pointer, and the contents of the array resolved_path are undefined, and errno is set to indicate the error. In this case, "undefined" seems to mean that it's acting like basename(3) in that it chops off the last path component. Thus, any brick that is a *sibling* of the one that we're trying to use is treated as its parent, causing the spurious failures. In my case I had this: gfs2:/export/sdb/dht was already part of a volume gfs2:/export/sdb/afr couldn't be used for a new volume Our use of realpath(3) for non-local paths is highly suspicious in general, but I'm not fixing that right now. In a moment I'll post a patch just to avoid the use of the undefined result.
http://review.gluster.org/#change,4201
CHANGE: http://review.gluster.org/4203 (glusterd: brick path availability check only for local bricks) merged in master by Vijay Bellur (vbellur)
CHANGE: http://review.gluster.org/4201 (glusterd: fix use of undefined realpath(3) result) merged in master by Vijay Bellur (vbellur)
*** Bug 875412 has been marked as a duplicate of this bug. ***