Bug 1473129
Summary: | dht/rebalance: Improve rebalance crawl performance | ||
---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Susant Kumar Palai <spalai> |
Component: | distribute | Assignee: | Susant Kumar Palai <spalai> |
Status: | CLOSED CURRENTRELEASE | QA Contact: | |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | 3.10 | CC: | bugs |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | glusterfs-3.10.5 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | 1439571 | Environment: | |
Last Closed: | 2017-08-21 13:41:22 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1439571 | ||
Bug Blocks: |
Description
Susant Kumar Palai
2017-07-20 05:28:12 UTC
REVIEW: https://review.gluster.org/17830 (dht/rebalance: Crawler performance improvement) posted (#1) for review on release-3.10 by Susant Palai (spalai) REVIEW: https://review.gluster.org/17830 (dht/rebalance: Crawler performance improvement) posted (#2) for review on release-3.10 by Susant Palai (spalai) REVIEW: https://review.gluster.org/17830 (dht/rebalance: Crawler performance improvement) posted (#3) for review on release-3.10 by Shyamsundar Ranganathan (srangana) REVIEW: https://review.gluster.org/17830 (dht/rebalance: Crawler performance improvement) posted (#4) for review on release-3.10 by Shyamsundar Ranganathan (srangana) COMMIT: https://review.gluster.org/17830 committed in release-3.10 by Shyamsundar Ranganathan (srangana) ------ commit a97d446dd52d739cbfc2e7689155ca1f6641ced6 Author: Susant Palai <spalai> Date: Mon Nov 7 12:00:13 2016 +0530 dht/rebalance: Crawler performance improvement The job of the crawler in rebalance is to fetch files from each local subvolume and push them to migration queue if it is eligible for migration. And we do a lookup on the entries received to figure out the eligibilty. Since, the lookup done is on a local subvolume we receive linkto files and regular files as well. This requires us to do two lookups. first: do a lookup on the file to figure out whether it is a linkto file second: do a lookup on the file to figure out if it should be migrated Note: The migrator thread also does one lookup for the file before migration. Optimization: Remove the lookup done by the crawler. Offload these task to the migrator threads. For linkto file verification get the stat and xattr information from readdirp. So in total we have one lookup instead of three for each entry. Performance numbers: Create two node, two brick setup. Created 100000 files. And started rebalance. Since, there is no add-brick, no files will be migrated and we will get the crawler performance. Without patch: [root@gprfs039 ~]# grs Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 0 0Bytes 50070 0 0 completed 0:0:48 server2 0 0Bytes 49930 0 0 completed 0:0:44 volume rebalance: test1: success Total: 48 seconds WiththecurrentPatch: [root@gprfs039 mnt]# gluster v rebalance test1 status Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 0 0Bytes 50070 0 0 completed 0:0:12 server2 0 0Bytes 49930 0 0 completed 0:0:12 volume rebalance: test1: success Total: 12 seconds That's 4X speed gain. :) > Updates glusterfs#155 > Change-Id: Idc8e5b366e76c54aa40d698876ae62fe1630b6cc > BUG: 1439571 > Signed-off-by: Susant Palai <spalai> > Reviewed-on: https://review.gluster.org/15781 > Smoke: Gluster Build System <jenkins.org> > NetBSD-regression: NetBSD Build System <jenkins.org> > CentOS-regression: Gluster Build System <jenkins.org> > Reviewed-by: Raghavendra G <rgowdapp> Updates glusterfs#155 Change-Id: Idc8e5b366e76c54aa40d698876ae62fe1630b6cc BUG: 1473129 Signed-off-by: Susant Palai <spalai> Reviewed-on: https://review.gluster.org/17830 CentOS-regression: Gluster Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> Reviewed-by: Shyamsundar Ranganathan <srangana> This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.5, please open a new bug report. glusterfs-3.10.5 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2017-August/000079.html [2] https://www.gluster.org/pipermail/gluster-users/ |