+++ This bug was initially created as a clone of Bug #1418900 +++ Description of problem: ----------------------- There are some volume options that needs to be added to virt file network.remote-dio=off features.shard-block-size=4MB performance.low-prio-threads=32 cluster.locking-scheme=granular cluster.shd-max-threads=8 cluster.shd-wait-qlength=10000 performance.strict-o-direct=on user.cifs=off nfs.disable=on cluster.use-compound-fops=on Version-Release number of selected component (if applicable): ------------------------------------------------------------- mainline How reproducible: ----------------- Not applicable Steps to Reproduce: ------------------- Not applicable Actual results: --------------- Not applicable Expected results: ----------------- Few more options added to virt file Additional info: ---------------- granular-entry-heal also needs to be enabled on the volume, but this option could only be added unless the volume is started. --- Additional comment from Worker Ant on 2017-02-09 05:35:12 EST --- REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#1) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Worker Ant on 2017-02-10 01:32:35 EST --- REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#2) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Worker Ant on 2017-02-10 01:42:22 EST --- REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#3) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Worker Ant on 2017-02-16 23:30:37 EST --- REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#4) for review on master by Krutika Dhananjay (kdhananj) --- Additional comment from Krutika Dhananjay on 2017-02-17 05:15:32 EST --- Why are each of these individual options added? 1. performance.low-prio-threads=32 VM use cases involve more of reads and writes than any other operations. And both of these are categorized as 'low_prio' operations by io-threads. Setting low-prio-threads to 32 saw some improvement in performance in initial testing done by Paul Cuzner. 2. cluster.locking-scheme=granular This is needed for more granular locking during entry self-heal so that parallel heal-info doesn't hang until heals are complete. 3. features.shard=on This is needed to reduce heal-times of VM image files which are large files whose size can range anywhere from a few GBs to TBs or even PBs. 4. cluster.shd-max-threads=8, cluster.shd-wait-qlength=10000 These are to leverage multi-threaded self-heal feature to get better heal performance and minimise the effect of downtime on VM file consistency and availability. 5. user.cifs=off We support access to files in virt use case only through libgfapi and FUSE. So we explicitly disable other methods of access such as cifs. --- Additional comment from Worker Ant on 2017-02-17 05:44:01 EST --- COMMIT: https://review.gluster.org/16577 committed in master by Niels de Vos (ndevos) ------ commit 932ff80b57846ed77b66776ade575a0b2231183a Author: Krutika Dhananjay <kdhananj> Date: Thu Feb 9 15:56:04 2017 +0530 extras: Add more options to group virt Apart from some of the option configurations already listed in the group-virt.example file, we also recommend that the users set certain other options added by this patch for VM use-case. This also helps Gluster-oVirt users in configuring virt options for new volumes at the click of a button as opposed to setting them manually through volume-set command. Change-Id: I8524e8d8a06bbbb0b9247571706e786410013b41 BUG: 1418900 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: https://review.gluster.org/16577 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Sahina Bose <sabose> Reviewed-by: Niels de Vos <ndevos>
REVIEW: https://review.gluster.org/16683 (extras: Add more options to group virt) posted (#1) for review on release-3.8 by Krutika Dhananjay (kdhananj)
With this patch, we have introduced 5 more options to group virt (VM use cases) for optimal performance. Updating to the glusterfs version containing this patch won't automatically set these newer options on already existing volumes that have group virt configured. The changes take effect only when post-upgrade a volume-set of group virt is performed. For already existing volumes the users may execute the following five commands, if not already set: gluster volume set <VOL> performance.low-prio-threads 32 gluster volume set <VOL> cluster.locking-scheme granular gluster volume set <VOL> features.shard on gluster volume set <VOL> cluster.shd-max-threads 8 gluster volume set <VOL> cluster.shd-wait-qlength 10000 gluster volume set <VOL> user.cifs off Of these my assumption is that features.shard would already have been set on the volume even before the upgrade, in which case the third volume-set command above may be skipped. -Krutika
REVIEW: https://review.gluster.org/16683 (extras: Add more options to group virt) posted (#2) for review on release-3.8 by Krutika Dhananjay (kdhananj)
Niels, As requested, Ive put down the commands that may need to be individually executed on existing volumes in comment #3. Could you review the comment once and merge the patch if everything looks ok? -Krutika
COMMIT: https://review.gluster.org/16683 committed in release-3.8 by Niels de Vos (ndevos) ------ commit fec404ce6feec3dcd1b4910cf791e46a4bfb6a9a Author: Krutika Dhananjay <kdhananj> Date: Thu Feb 9 15:56:04 2017 +0530 extras: Add more options to group virt Backport of: https://review.gluster.org/16577 Apart from some of the option configurations already listed in the group-virt.example file, we also recommend that the users set certain other options added by this patch for VM use-case. This also helps Gluster-oVirt users in configuring virt options for new volumes at the click of a button as opposed to setting them manually through volume-set command. Change-Id: I879d779a4829d7163de4df9ff80d15296130af8a BUG: 1424934 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: https://review.gluster.org/16683 NetBSD-regression: NetBSD Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu> Reviewed-by: Niels de Vos <ndevos>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.10, please open a new bug report. glusterfs-3.8.10 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2017-March/000068.html [2] https://www.gluster.org/pipermail/gluster-users/