Description of problem: In the course of investigating what seemed to be better-than-possible performance with replication (replicated performance was better than either replica locally) I found that write-behind does not fully honor write-behind. It sets the window size to zero, but only prevents requests from being coalesced; they're still queued and executed asynchronously. How reproducible: Issue synchronous writes. Observe their progress through the system in a debugger, or observe total performance that's not possible with proper O_SYNC handling. Actual results: Asynchronous write processing and inflated performance. If there's a failure, data that the user thought written might be lost (hence the high severity). Expected results: Synchronous write processing and protection of user data.
NB: it is possible to avoid this problem thus. gluster volume set xxx performance.write-behind off However, foregoing write-behind on an entire volume to get correct O_SYNC behavior on one file is unacceptable.
(In reply to comment #0) > replica locally) I found that write-behind does not fully honor write-behind ..does not fully honor O_SYNC...
CHANGE: http://review.gluster.com/2626 (Honor O_SYNC etc. properly.) merged in master by Anand Avati (avati)
Verified with 3.3.0qa45