Bug 916372
| Summary: | NFS3 stable writes are very slow | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Anand Avati <aavati> |
| Component: | nfs | Assignee: | Anand Avati <aavati> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | mainline | CC: | chrisw, gluster-bugs, jdarcy, kaushal, vagarwal |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | glusterfs-3.4.0 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-07-24 17:32:51 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 952693 | ||
Patch posted: http://review.gluster.org/4591 output of gluster volume profile with the patch: 0.07 146.20 us 53.00 us 333.00 us 5 FINODELK 2.70 13301.00 us 10449.00 us 16153.00 us 2 XATTROP 5.03 24809.00 us 10529.00 us 39089.00 us 2 FXATTROP 14.20 139982.00 us 139982.00 us 139982.00 us 1 TRUNCATE 77.85 749.71 us 601.00 us 1282.00 us 1024 WRITE CHANGE: http://review.gluster.org/4591 (nfs: handle stable write with @flags rather than fsync()) merged in master by Anand Avati (avati) REVIEW: http://review.gluster.org/4738 (storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev()) posted (#1) for review on master by Anand Avati (avati) COMMIT: http://review.gluster.org/4738 committed in master by Anand Avati (avati) ------ commit fdb05c6f84054ca640e3da1c19ea7d536d2751e0 Author: Anand Avati <avati> Date: Mon Mar 25 12:18:13 2013 -0700 storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev() Historic bug - posix_writev() has been inspecting pfd->flushwrites for performing fsync() after write, instead of @flags for O_SYNC|O_DSYNC. pfd->flushwrites was never set anywhere and is unused completely. This is behavior from the time before anonymous FD where open() had @wbflags param. This is a leftover from that cleanup. Change-Id: Id9bfe562a60db4eb3bd0a7705bdba91f2df2f3ec BUG: 916372 Signed-off-by: Anand Avati <avati> Reviewed-on: http://review.gluster.org/4738 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu> REVIEW: http://review.gluster.org/4962 (storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev()) posted (#1) for review on release-3.4 by Vijay Bellur (vbellur) COMMIT: http://review.gluster.org/4962 committed in release-3.4 by Anand Avati (avati) ------ commit cce370f6d2c1d3bfaf1d772ebe5d6a01f761016f Author: Anand Avati <avati> Date: Mon Mar 25 12:18:13 2013 -0700 storage/posix: honor O_SYNC and O_DSYNC sent in @flags of writev() Historic bug - posix_writev() has been inspecting pfd->flushwrites for performing fsync() after write, instead of @flags for O_SYNC|O_DSYNC. pfd->flushwrites was never set anywhere and is unused completely. This is behavior from the time before anonymous FD where open() had @wbflags param. This is a leftover from that cleanup. Change-Id: Id9bfe562a60db4eb3bd0a7705bdba91f2df2f3ec BUG: 916372 Signed-off-by: Anand Avati <avati> Reviewed-on: http://review.gluster.org/4738 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu> Reviewed-on: http://review.gluster.org/4962 |
stable writes in NFS is implemented as a regular write followed by an fsync(). The following is seen in volume profile: 1.48 3038.06 us 41.00 us 59437703.00 us 34483 FINODELK 2.69 11529.24 us 212.00 us 388617.00 us 16538 WRITE 10.87 47896.16 us 51.00 us 640614274.00 us 16070 INODELK 12.24 52427.81 us 949.00 us 505374.00 us 16536 FSYNC 72.72 209020.42 us 4771.00 us 1111574.00 us 24641 FXATTROP This logic completely breaks the eager-locking optimizations for no good reason, and must instead just use the @flags parameter of the write fop to set O_SYNC bit.