Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1428055

Summary: dht/rebalance: Increase maximum read block size from 128 KB to 1 MB
Product: [Community] GlusterFS Reporter: Vijay Bellur <vbellur>
Component: distributeAssignee: Nithya Balachandran <nbalacha>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: mainlineCC: bugs
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.11.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1442943 (view as bug list) Environment:
Last Closed: 2017-05-30 18:46:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1442943    

Description Vijay Bellur 2017-03-01 19:05:02 UTC
dht/rebalance: Increase maximum read block size from 128 KB to 1 MB

Summary:
- The maximum block size, `DHT_REBALANCE_BLKSIZE`, is set to 128 KB.
- As a result, migrating files in the megabytes to gigabytes can take much longer than necessary.
- Some preliminary results by bumping the blocksize:

With 128 KB:
  [2016-08-04 11:40:19.251167] I [MSGID: 109028] [dht-rebalance.c:2196:gf_defrag_status_get] 0-glusterfs: Rebalance is completed. Time taken is 15.00 secs
  [2016-08-04 11:40:19.251189] I [MSGID: 109028] [dht-rebalance.c:2200:gf_defrag_status_get] 0-glusterfs: Files migrated: 49, size: 2569011200, lookups: 149, failures: 0, skipped: 0

With 1 MB:
  [2016-08-04 11:41:21.093662] I [MSGID: 109028] [dht-rebalance.c:2196:gf_defrag_status_get] 0-glusterfs: Rebalance is completed. Time taken is 7.00 secs
  [2016-08-04 11:41:21.093687] I [MSGID: 109028] [dht-rebalance.c:2200:gf_defrag_status_get] 0-glusterfs: Files migrated: 49, size: 2569011200, lookups: 149, failures: 0, skipped: 0

- This is a cherry-pick of D3670927 to 3.8.

Test Plan: Tested rebalance on devserver.

Reviewed By: dph, rwareing

Change-Id: Ide2edbf87ef9ae2b32a03f189c57b63e2f233fc8
Signed-off-by: Shreyas Siravara <sshreyas>
Reviewed-on: http://review.gluster.org/16078
CentOS-regression: Gluster Build System <jenkins.org>
NetBSD-regression: NetBSD Build System <jenkins.org>
Smoke: Gluster Build System <jenkins.org>
Reviewed-by: Kevin Vigor <kvigor>

Comment 1 Worker Ant 2017-03-01 19:52:19 UTC
REVIEW: https://review.gluster.org/16797 (dht/rebalance: Increase maximum read block size from 128 KB to 1 MB) posted (#1) for review on master by Vijay Bellur (vbellur)

Comment 2 Worker Ant 2017-03-02 18:16:16 UTC
REVIEW: https://review.gluster.org/16797 (dht/rebalance: Increase maximum read block size from 128 KB to 1 MB) posted (#2) for review on master by Vijay Bellur (vbellur)

Comment 3 Worker Ant 2017-03-02 23:54:45 UTC
COMMIT: https://review.gluster.org/16797 committed in master by Jeff Darcy (jdarcy) 
------
commit 351638ba61d2451d97363e71adb9a9ce7d5b9c85
Author: Shreyas Siravara <sshreyas>
Date:   Thu Aug 4 11:44:53 2016 -0700

    dht/rebalance: Increase maximum read block size from 128 KB to 1 MB
    
    Summary:
    - The maximum block size, `DHT_REBALANCE_BLKSIZE`, is set to 128 KB.
    - As a result, migrating files in the megabytes to gigabytes can take much longer than necessary.
    - Some preliminary results by bumping the blocksize:
    
    With 128 KB:
      [2016-08-04 11:40:19.251167] I [MSGID: 109028] [dht-rebalance.c:2196:gf_defrag_status_get] 0-glusterfs: Rebalance is completed. Time taken is 15.00 secs
      [2016-08-04 11:40:19.251189] I [MSGID: 109028] [dht-rebalance.c:2200:gf_defrag_status_get] 0-glusterfs: Files migrated: 49, size: 2569011200, lookups: 149, failures: 0, skipped: 0
    
    With 1 MB:
      [2016-08-04 11:41:21.093662] I [MSGID: 109028] [dht-rebalance.c:2196:gf_defrag_status_get] 0-glusterfs: Rebalance is completed. Time taken is 7.00 secs
      [2016-08-04 11:41:21.093687] I [MSGID: 109028] [dht-rebalance.c:2200:gf_defrag_status_get] 0-glusterfs: Files migrated: 49, size: 2569011200, lookups: 149, failures: 0, skipped: 0
    
    - This is a cherry-pick of D3670927 to 3.8.
    
    Test Plan: Tested rebalance on devserver.
    
    Reviewed By: dph, rwareing
    
    Change-Id: Ide2edbf87ef9ae2b32a03f189c57b63e2f233fc8
    BUG: 1428055
    Signed-off-by: Shreyas Siravara <sshreyas>
    Reviewed-on: https://review.gluster.org/16797
    Tested-by: Vijay Bellur <vbellur>
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 4 Shyamsundar 2017-05-30 18:46:32 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.11.0, please open a new bug report.

glusterfs-3.11.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-May/000073.html
[2] https://www.gluster.org/pipermail/gluster-users/