Bug 916372 - NFS3 stable writes are very slow
Summary: NFS3 stable writes are very slow
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: nfs
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Anand Avati
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 952693
TreeView+ depends on / blocked
 
Reported: 2013-02-27 23:02 UTC by Anand Avati
Modified: 2015-09-01 23:06 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.4.0
Clone Of:
Environment:
Last Closed: 2013-07-24 17:32:51 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Anand Avati 2013-02-27 23:02:17 UTC
stable writes in NFS is implemented as a regular write followed by an fsync(). The following is seen in volume profile:

      1.48    3038.06 us      41.00 us 59437703.00 us          34483    FINODELK
      2.69   11529.24 us     212.00 us  388617.00 us          16538       WRITE
     10.87   47896.16 us      51.00 us 640614274.00 us          16070     INODELK
     12.24   52427.81 us     949.00 us  505374.00 us          16536       FSYNC
     72.72  209020.42 us    4771.00 us 1111574.00 us          24641    FXATTROP

This logic completely breaks the eager-locking optimizations for no good reason, and must instead just use the @flags parameter of the write fop to set O_SYNC bit.

Comment 1 Anand Avati 2013-02-27 23:06:31 UTC
Patch posted: http://review.gluster.org/4591

output of gluster volume profile with the patch:

      0.07     146.20 us      53.00 us     333.00 us              5    FINODELK
      2.70   13301.00 us   10449.00 us   16153.00 us              2     XATTROP
      5.03   24809.00 us   10529.00 us   39089.00 us              2    FXATTROP
     14.20  139982.00 us  139982.00 us  139982.00 us              1    TRUNCATE
     77.85     749.71 us     601.00 us    1282.00 us           1024       WRITE

Comment 2 Vijay Bellur 2013-03-21 02:52:46 UTC
CHANGE: http://review.gluster.org/4591 (nfs: handle stable write with @flags rather than fsync()) merged in master by Anand Avati (avati)

Comment 3 Vijay Bellur 2013-03-28 06:12:50 UTC
REVIEW: http://review.gluster.org/4738 (storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev()) posted (#1) for review on master by Anand Avati (avati)

Comment 4 Vijay Bellur 2013-03-29 19:08:02 UTC
COMMIT: http://review.gluster.org/4738 committed in master by Anand Avati (avati) 
------
commit fdb05c6f84054ca640e3da1c19ea7d536d2751e0
Author: Anand Avati <avati>
Date:   Mon Mar 25 12:18:13 2013 -0700

    storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev()
    
    Historic bug - posix_writev() has been inspecting pfd->flushwrites for
    performing fsync() after write, instead of @flags for O_SYNC|O_DSYNC.
    
    pfd->flushwrites was never set anywhere and is unused completely. This
    is behavior from the time before anonymous FD where open() had @wbflags
    param. This is a leftover from that cleanup.
    
    Change-Id: Id9bfe562a60db4eb3bd0a7705bdba91f2df2f3ec
    BUG: 916372
    Signed-off-by: Anand Avati <avati>
    Reviewed-on: http://review.gluster.org/4738
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

Comment 5 Anand Avati 2013-05-07 12:33:24 UTC
REVIEW: http://review.gluster.org/4962 (storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev()) posted (#1) for review on release-3.4 by Vijay Bellur (vbellur)

Comment 6 Anand Avati 2013-05-07 18:02:41 UTC
COMMIT: http://review.gluster.org/4962 committed in release-3.4 by Anand Avati (avati) 
------
commit cce370f6d2c1d3bfaf1d772ebe5d6a01f761016f
Author: Anand Avati <avati>
Date:   Mon Mar 25 12:18:13 2013 -0700

    storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev()
    
    Historic bug - posix_writev() has been inspecting pfd->flushwrites for
    performing fsync() after write, instead of @flags for O_SYNC|O_DSYNC.
    
    pfd->flushwrites was never set anywhere and is unused completely. This
    is behavior from the time before anonymous FD where open() had @wbflags
    param. This is a leftover from that cleanup.
    
    Change-Id: Id9bfe562a60db4eb3bd0a7705bdba91f2df2f3ec
    BUG: 916372
    Signed-off-by: Anand Avati <avati>
    Reviewed-on: http://review.gluster.org/4738
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    Reviewed-on: http://review.gluster.org/4962


Note You need to log in before you can comment on or make changes to this bug.