If a file is deleted while a brick is offline, and a file by the same name is subsequently created once the brick has been brought back online but not fully healed, the brick returns the old, pre-deletion GFID for the file, This results in lack of consensus about the GFID, which is bad, and usually manifests as EROFS. Reproduce by repeatedly untarring linux source tarball onto 2-way replicated volume while bringing the second replica server up and down every 30 seconds. Consistently fails within half an hour.
REVIEW: http://review.gluster.org/14153 (Fix race in file creation when brick was offline during delete.) posted (#1) for review on master by Kevin Vigor (kvigor)
This bug is being closed as GlusterFS-3.6 is nearing its End-Of-Life and only important security bugs will be fixed. If you still face this bug with the newer GlusterFS versions, please open a new bug.