Bug 988236 - Large File Sequential I/O: Poor performance from GlusterFS over SMB
Large File Sequential I/O: Poor performance from GlusterFS over SMB
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: samba (Show other bugs)
All Unspecified
high Severity high
: ---
: ---
Assigned To: Poornima G
Ben Turner
Depends On:
Blocks: 956495
  Show dependency treegraph
Reported: 2013-07-25 03:23 EDT by Jose A. Rivera
Modified: 2013-09-23 18:32 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-09-23 18:32:16 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jose A. Rivera 2013-07-25 03:23:31 EDT
This bug was originally found by Tim Wilkinson and Ben England.

When trying to do sequential I/O (reads and writes) on a large file on a GlusterFS-based SMB share, we were getting unusual and unacceptably poor performance. On a lead from Ben regarding some odd locking behaviour, Raghavendra Talur and Anand Avati went digging and discovered a one-line smb.conf fix that seems to dramatically improve the performance in question:

  kernel oplocks = no

Comparing against I/O on a large file on an XFS-based SMB share, we get the following numbers for the VFS/libgfapi Samba configuration (in contrast to the FUSE-based Samba configuration):

WRITES: libgfapi is at ~85% of XFS (while FUSE is ~48%)
READS:  libgfapi is at ~78% of XFS (while FUSE is ~88%)

We recommend implementing this configuration option right away in RHS.
Comment 2 Christopher R. Hertel 2013-08-05 18:00:41 EDT
In the original description, a simple one-line change to the smb.conf file is recommended.  This change should have been handled in another BZ, so the change may already be in place.

Please check to see whether the change is in place.  If not, please add

  kernel oplocks = no

to the smb.conf configuration.  Gluster does not support oplocks, so this is the correct setting.
Comment 3 Poornima G 2013-08-06 02:37:59 EDT
This fix is released as a part of RHS-2.1-20130806.n.0.
Comment 4 Ben Turner 2013-08-07 17:27:19 EDT
Verified on RHS-2.1-20130806.n.2-RHS-x86_64-DVD1.iso, the /etc/samba/smb.conf now has:

# ------------------------ RHS Performance Tunings -------------------------
# The following options improve RHS performance in the following respective
# scenarios:
# - Large file sequential I/O
# - Small file operations

        kernel oplocks = no
        stat cache = no

#============================ Share Definitions ==============================
Comment 5 Scott Haines 2013-09-23 18:32:16 EDT
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.