Bug 762011 (GLUSTER-279)

Summary: File written with booster results in self-heal after dd exits
Product: [Community] GlusterFS Reporter: Shehjar Tikoo <shehjart>
Component: boosterAssignee: Shehjar Tikoo <shehjart>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: low    
Version: mainlineCC: gluster-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: RTP Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Shehjar Tikoo 2009-09-23 04:16:43 UTC
The solution lies in either making the early unwind in replicate to be disabled through an option or by trapping exit() in booster and ensuring that libglusterfsclient waits for all call frames to return before destroying the glusterfs context state.

Comment 1 Shehjar Tikoo 2009-09-23 07:15:06 UTC
Suppose we're writing a file through dd with booster preloaded booster. Now if we have the same volfile  as being used by booster, mounted through FUSE. As soon as the dd file writing finishes, if do an ls on the mount point, a self-heal gets triggered. This happens consistently.

The problem has been narrowed down to the replicate *feature*..(:P) of unwinding a call before the xattrs have been set to a sane value. This means that when the close syscall returns control to booster, there is no guarantee that the file has been written sanely on all replicate children. Right after the close returns, dd exits, giving no opportunity to replicate to set the xattrs in the background.

Next, after dd exits, doing an ls makes replicate realize the inconsistent xattrs and so triggers a self-heal.

Comment 2 Anand Avati 2009-09-24 05:19:00 UTC
PATCH: http://patches.gluster.com/patch/1469 in master (iobuf, logging: Expose existing functions as required)

Comment 3 Anand Avati 2009-09-24 05:19:04 UTC
PATCH: http://patches.gluster.com/patch/1470 in master (libglusterfsclient: Clean up fini and umount code paths)

Comment 4 Anand Avati 2009-09-24 05:19:08 UTC
PATCH: http://patches.gluster.com/patch/1471 in master (libglusterfsclient: Wait for all call pools to be destroyed)

Comment 5 Anand Avati 2009-09-24 05:19:15 UTC
PATCH: http://patches.gluster.com/patch/1473 in master (booster: Cleanup booster_cleanup and register for atexit)

Comment 6 Anand Avati 2009-09-24 05:19:34 UTC
PATCH: http://patches.gluster.com/patch/1476 in release-2.0 (iobuf, logging: Expose existing functions as required)

Comment 7 Anand Avati 2009-09-24 05:19:38 UTC
PATCH: http://patches.gluster.com/patch/1477 in release-2.0 (libglusterfsclient: Clean up fini and umount code paths)

Comment 8 Anand Avati 2009-09-24 05:19:42 UTC
PATCH: http://patches.gluster.com/patch/1478 in release-2.0 (libglusterfsclient: Wait for all call pools to be destroyed)

Comment 9 Anand Avati 2009-09-24 05:19:49 UTC
PATCH: http://patches.gluster.com/patch/1480 in release-2.0 (booster: Cleanup booster_cleanup and register for atexit)