Bug 1424934 - Include few more options in virt file
Summary: Include few more options in virt file
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: 3.8
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Krutika Dhananjay
QA Contact:
URL:
Whiteboard:
Depends On: 1418900
Blocks: 1418901 1424931 1424932
TreeView+ depends on / blocked
 
Reported: 2017-02-20 06:05 UTC by Krutika Dhananjay
Modified: 2017-03-18 10:52 UTC (History)
4 users (show)

Fixed In Version: glusterfs-3.8.10
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1418900
Environment:
Last Closed: 2017-03-18 10:52:28 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Krutika Dhananjay 2017-02-20 06:05:45 UTC
+++ This bug was initially created as a clone of Bug #1418900 +++

Description of problem:
-----------------------
There are some volume options that needs to be added to virt file
network.remote-dio=off
features.shard-block-size=4MB
performance.low-prio-threads=32
cluster.locking-scheme=granular
cluster.shd-max-threads=8
cluster.shd-wait-qlength=10000
performance.strict-o-direct=on
user.cifs=off
nfs.disable=on
cluster.use-compound-fops=on

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
mainline

How reproducible:
-----------------
Not applicable

Steps to Reproduce:
-------------------
Not applicable

Actual results:
---------------
Not applicable

Expected results:
-----------------
Few more options added to virt file

Additional info:
----------------
granular-entry-heal also needs to be enabled on the volume, but this option could only be added unless the volume is started.

--- Additional comment from Worker Ant on 2017-02-09 05:35:12 EST ---

REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Worker Ant on 2017-02-10 01:32:35 EST ---

REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#2) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Worker Ant on 2017-02-10 01:42:22 EST ---

REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#3) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Worker Ant on 2017-02-16 23:30:37 EST ---

REVIEW: https://review.gluster.org/16577 (extras: Add more options to group virt) posted (#4) for review on master by Krutika Dhananjay (kdhananj)

--- Additional comment from Krutika Dhananjay on 2017-02-17 05:15:32 EST ---

Why are each of these individual options added?

1. performance.low-prio-threads=32
VM use cases involve more of reads and writes than any other operations. And both of these are categorized as 'low_prio' operations by io-threads. Setting low-prio-threads to 32 saw some improvement in performance in initial testing done by Paul Cuzner.

2. cluster.locking-scheme=granular
This is needed for more granular locking during entry self-heal so that parallel heal-info doesn't hang until heals are complete.

3. features.shard=on
This is needed to reduce heal-times of VM image files which are large files whose size can range anywhere from a few GBs to TBs or even PBs.

4. cluster.shd-max-threads=8, cluster.shd-wait-qlength=10000
These are to leverage multi-threaded self-heal feature to get better heal performance and minimise the effect of downtime on VM file consistency and availability.

5. user.cifs=off
We support access to files in virt use case only through libgfapi and FUSE. So we explicitly disable other methods of access such as cifs.

--- Additional comment from Worker Ant on 2017-02-17 05:44:01 EST ---

COMMIT: https://review.gluster.org/16577 committed in master by Niels de Vos (ndevos) 
------
commit 932ff80b57846ed77b66776ade575a0b2231183a
Author: Krutika Dhananjay <kdhananj>
Date:   Thu Feb 9 15:56:04 2017 +0530

    extras: Add more options to group virt
    
    Apart from some of the option configurations already listed
    in the group-virt.example file, we also recommend that the
    users set certain other options added by this patch for
    VM use-case. This also helps Gluster-oVirt users in configuring
    virt options for new volumes at the click of a button as opposed
    to setting them manually through volume-set command.
    
    Change-Id: I8524e8d8a06bbbb0b9247571706e786410013b41
    BUG: 1418900
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: https://review.gluster.org/16577
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Sahina Bose <sabose>
    Reviewed-by: Niels de Vos <ndevos>

Comment 1 Worker Ant 2017-02-20 06:06:51 UTC
REVIEW: https://review.gluster.org/16683 (extras: Add more options to group virt) posted (#1) for review on release-3.8 by Krutika Dhananjay (kdhananj)

Comment 2 Krutika Dhananjay 2017-02-21 09:38:12 UTC
With this patch, we have introduced 5 more options to group virt (VM use cases) for optimal performance.

Updating to the glusterfs version containing this patch won't automatically set these newer options on already existing volumes that have group virt configured. The changes take effect only when post-upgrade a volume-set of group virt is performed.
For already existing volumes the users may execute the following five commands, if not already set:

gluster volume set <VOL> performance.low-prio-threads 32
gluster volume set <VOL> cluster.locking-scheme granular
gluster volume set <VOL> features.shard on
gluster volume set <VOL> cluster.shd-max-threads 8
gluster volume set <VOL> cluster.shd-wait-qlength 10000
gluster volume set <VOL> user.cifs off

Of these my assumption is that features.shard would already have been set on the volume even before the upgrade, in which case the third volume-set command above may be skipped.

-Krutika

Comment 3 Worker Ant 2017-02-22 07:08:45 UTC
REVIEW: https://review.gluster.org/16683 (extras: Add more options to group virt) posted (#2) for review on release-3.8 by Krutika Dhananjay (kdhananj)

Comment 4 Krutika Dhananjay 2017-02-24 06:20:17 UTC
Niels,

As requested, Ive put down the commands that may need to be individually executed on existing volumes in comment #3. Could you review the comment once and merge the patch if everything looks ok?

-Krutika

Comment 5 Worker Ant 2017-03-12 10:11:06 UTC
COMMIT: https://review.gluster.org/16683 committed in release-3.8 by Niels de Vos (ndevos) 
------
commit fec404ce6feec3dcd1b4910cf791e46a4bfb6a9a
Author: Krutika Dhananjay <kdhananj>
Date:   Thu Feb 9 15:56:04 2017 +0530

    extras: Add more options to group virt
    
            Backport of: https://review.gluster.org/16577
    
    Apart from some of the option configurations already listed
    in the group-virt.example file, we also recommend that the
    users set certain other options added by this patch for
    VM use-case. This also helps Gluster-oVirt users in configuring
    virt options for new volumes at the click of a button as opposed
    to setting them manually through volume-set command.
    
    Change-Id: I879d779a4829d7163de4df9ff80d15296130af8a
    BUG: 1424934
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: https://review.gluster.org/16683
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    Reviewed-by: Niels de Vos <ndevos>

Comment 6 Niels de Vos 2017-03-18 10:52:28 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.10, please open a new bug report.

glusterfs-3.8.10 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-March/000068.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.