Bug 1250803 - Perf: Metadata operation(ls -l) performance regression.
Perf: Metadata operation(ls -l) performance regression.
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: replicate (Show other bugs)
mainline
x86_64 Linux
high Severity high
: ---
: ---
Assigned To: Pranith Kumar K
:
Depends On: 1250241
Blocks: 1287531 1315144
  Show dependency treegraph
 
Reported: 2015-08-06 00:03 EDT by Pranith Kumar K
Modified: 2016-06-16 09:28 EDT (History)
13 users (show)

See Also:
Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1250241
: 1287531 1315144 (view as bug list)
Environment:
Last Closed: 2016-06-16 09:28:57 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 1 Anand Avati 2015-08-06 00:09:43 EDT
REVIEW: http://review.gluster.org/11846 (cluster/afr: make consistent metadata less restrictive) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 2 Anand Avati 2015-08-06 00:19:26 EDT
REVIEW: http://review.gluster.org/11846 (cluster/afr: make consistent metadata less restrictive) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 3 Ben Turner 2015-09-17 12:00:47 EDT
Changing back to assigned as the patch was abandoned.
Comment 4 Vijay Bellur 2015-10-30 02:58:59 EDT
REVIEW: http://review.gluster.org/12467 (glusterd / index : Readdir performance improvement) posted (#1) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 5 Anuradha 2015-10-30 03:18:12 EDT
Patch2:
http://review.gluster.org/#/c/12468/
Comment 6 Vijay Bellur 2015-11-12 05:28:35 EST
REVIEW: http://review.gluster.org/12468 (features / index : Readdirp performance improvement) posted (#6) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 7 Vijay Bellur 2015-11-12 09:13:38 EST
REVIEW: http://review.gluster.org/12468 (features / index : Readdirp performance improvement) posted (#7) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 8 Vijay Bellur 2015-11-17 03:32:27 EST
REVIEW: http://review.gluster.org/12467 (glusterd/afr : Readdirp performance improvement) posted (#6) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 9 Vijay Bellur 2015-11-17 03:32:33 EST
REVIEW: http://review.gluster.org/12595 (mgmt/gluster: Handle tier brick volgen) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 10 Vijay Bellur 2015-11-17 04:21:23 EST
REVIEW: http://review.gluster.org/12595 (mgmt/gluster: Handle tier brick volgen) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 11 Vijay Bellur 2015-11-17 04:21:34 EST
REVIEW: http://review.gluster.org/12467 (glusterd/afr : Readdirp performance improvement) posted (#7) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 12 Vijay Bellur 2015-11-18 00:02:53 EST
REVIEW: http://review.gluster.org/12595 (mgmt/gluster: Handle tier brick volgen) posted (#3) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 13 Vijay Bellur 2015-11-18 00:03:01 EST
REVIEW: http://review.gluster.org/12467 (glusterd/afr : Readdirp performance improvement) posted (#8) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 14 Vijay Bellur 2015-11-18 08:15:23 EST
REVIEW: http://review.gluster.org/12595 (mgmt/gluster: Handle tier brick volgen) posted (#4) for review on master by Pranith Kumar Karampuri (pkarampu@redhat.com)
Comment 15 Vijay Bellur 2015-11-18 08:17:35 EST
COMMIT: http://review.gluster.org/12467 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit cb76371a56eb90a6f2ac126c876d7a2dc203e16c
Author: Anuradha Talur <atalur@redhat.com>
Date:   Wed Oct 21 16:49:49 2015 +0530

    glusterd/afr : Readdirp performance improvement
    
    Add xlator options to index xlator with xattrs that
    it needs to keep track of.
    
    Change-Id: If818673be5e626f77e65cc3a340f8cdd624179c2
    BUG: 1250803
    Signed-off-by: Anuradha Talur <atalur@redhat.com>
    Reviewed-on: http://review.gluster.org/12467
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 16 Vijay Bellur 2015-11-19 04:06:14 EST
COMMIT: http://review.gluster.org/12595 committed in master by Kaushal M (kaushal@redhat.com) 
------
commit db506cec11cdb2235aac7b6a896b4071bdcc6ae3
Author: Pranith Kumar K <pkarampu@redhat.com>
Date:   Tue Nov 17 14:00:10 2015 +0530

    mgmt/gluster: Handle tier brick volgen
    
    Index xlator watches only some xattrs based on type of volume. i.e.
    disperse/afr. When the volume becomes tiered then index is not adding these
    options in the volfile leading to no maintenance of indices. Thus no
    proactive self-heals. With this fix, we write brick volfiles considering
    the type of volume they belong to.
    
    BUG: 1250803
    Change-Id: Ibe8f2d4ad5cb350306ab7ca0753e0f9a40b96a26
    Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
    Reviewed-on: http://review.gluster.org/12595
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Kaushal M <kaushal@redhat.com>
Comment 17 Vijay Bellur 2015-11-19 07:30:03 EST
REVIEW: http://review.gluster.org/12468 (features / index : Readdirp performance improvement) posted (#8) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 18 Vijay Bellur 2015-11-19 08:41:37 EST
REVIEW: http://review.gluster.org/12468 (features / index : Readdirp performance improvement) posted (#9) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 19 Vijay Bellur 2015-11-20 05:08:49 EST
REVIEW: http://review.gluster.org/12468 (features/index : Readdirp performance improvement) posted (#10) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 20 Vijay Bellur 2015-11-23 04:16:34 EST
REVIEW: http://review.gluster.org/12468 (features/index : Readdirp performance improvement) posted (#11) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 21 Vijay Bellur 2015-11-24 22:06:06 EST
COMMIT: http://review.gluster.org/12468 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit 5172bcb757553e05b18208d7726517f1ec4da837
Author: Anuradha Talur <atalur@redhat.com>
Date:   Thu Nov 12 19:41:33 2015 +0530

    features/index : Readdirp performance improvement
    
    Things done :
    Index now maintains two different directories :
    1) for pending xattrs (trusted.afr.volname-client-n,
                           trusted.ec.dirty)
    
    2) for on going xattrs (trusted.afr.dirty)
    
    Based on the xattr, index decides the directory
    to add/del entry in/from.
    
    Change-Id: Ie04c02ea2d862cf80426a871a9a1e80b0773d9fd
    BUG: 1250803
    Signed-off-by: Anuradha Talur <atalur@redhat.com>
    Reviewed-on: http://review.gluster.org/12468
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 22 Vijay Bellur 2015-11-26 09:32:54 EST
REVIEW: http://review.gluster.org/12507 (cluster/afr : Readdirp performance enhancement) posted (#4) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 23 Vijay Bellur 2015-11-30 23:24:09 EST
COMMIT: http://review.gluster.org/12507 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit f2c52ae206f309ec636a299a76849c843c8ab83d
Author: Anuradha Talur <atalur@redhat.com>
Date:   Thu Nov 12 19:45:10 2015 +0530

    cluster/afr : Readdirp performance enhancement
    
    Things done :
    1) during lookup and inode_refresh as part of read_txn,
    request is sent to detect if heal is required or not.
    
    2) If heal is required, be conservative in setting the
    readdirp entry inodes to NULL, otherwise don't be.
    
    3) Self-heal-daemon now crawls both indices/xattrop
    and indices/dirty directory while healing.
    
    Change-Id: Ic4a4da63fb7e0726eab5f341a200859b29cf7eb7
    BUG: 1250803
    Signed-off-by: Anuradha Talur <atalur@redhat.com>
    Reviewed-on: http://review.gluster.org/12507
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 24 Vijay Bellur 2015-12-01 06:17:17 EST
REVIEW: http://review.gluster.org/12658 (heal : Changed heal info to process all indices directories) posted (#5) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 25 Vijay Bellur 2015-12-01 06:51:03 EST
REVIEW: http://review.gluster.org/12658 (heal : Changed heal info to process all indices directories) posted (#6) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 26 Vijay Bellur 2015-12-02 11:52:08 EST
REVIEW: http://review.gluster.org/12658 (heal : Changed heal info to process all indices directories) posted (#7) for review on master by Vijay Bellur (vbellur@redhat.com)
Comment 27 Vijay Bellur 2015-12-02 19:00:30 EST
COMMIT: http://review.gluster.org/12658 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit 9c378026e9561595586a817fee0b439e2c863a22
Author: Anuradha Talur <atalur@redhat.com>
Date:   Mon Nov 23 15:56:28 2015 +0530

    heal : Changed heal info to process all indices directories
    
    Change-Id: Ida863844e14309b6526c1b8434273fbf05c410d2
    BUG: 1250803
    Signed-off-by: Anuradha Talur <atalur@redhat.com>
    Reviewed-on: http://review.gluster.org/12658
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Comment 28 Vijay Bellur 2015-12-08 06:37:36 EST
REVIEW: http://review.gluster.org/12910 (features/index : Prevent logging due to NULL dict) posted (#1) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 29 Vijay Bellur 2015-12-08 11:16:28 EST
REVIEW: http://review.gluster.org/12910 (features/index : Prevent logging due to NULL dict) posted (#2) for review on master by Vijay Bellur (vbellur@redhat.com)
Comment 30 Vijay Bellur 2015-12-08 16:51:46 EST
REVIEW: http://review.gluster.org/12910 (features/index : Prevent logging due to NULL dict) posted (#3) for review on master by Vijay Bellur (vbellur@redhat.com)
Comment 31 Vijay Bellur 2015-12-09 04:31:47 EST
REVIEW: http://review.gluster.org/12919 (heal : Do not print heal count on ENOTCONN) posted (#1) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 32 Vijay Bellur 2015-12-09 07:09:41 EST
REVIEW: http://review.gluster.org/12919 (heal : Do not print heal count on ENOTCONN) posted (#2) for review on master by Anuradha Talur (atalur@redhat.com)
Comment 33 Vijay Bellur 2015-12-11 02:11:17 EST
COMMIT: http://review.gluster.org/12910 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit ddf5957d3e4e09ed750f68447492796af61f08e4
Author: Anuradha Talur <atalur@redhat.com>
Date:   Tue Dec 8 17:02:23 2015 +0530

    features/index : Prevent logging due to NULL dict
    
    Added a check for non-NULLness of dict before
    performing dict_foreach_match on it.
    
    Change-Id: I77d83559934006425ed33745b8a244b2f5d90cb1
    BUG: 1250803
    Signed-off-by: Anuradha Talur <atalur@redhat.com>
    Reviewed-on: http://review.gluster.org/12910
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 34 Vijay Bellur 2015-12-15 04:58:28 EST
REVIEW: http://review.gluster.org/12971 (features/index : Prevent logging due to NULL dict) posted (#1) for review on release-3.7 by Anuradha Talur (atalur@redhat.com)
Comment 35 Vijay Bellur 2015-12-20 23:50:01 EST
COMMIT: http://review.gluster.org/12919 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit a99bbccbb5e3087b9b8b687ee2be8887eeac0d08
Author: Anuradha Talur <atalur@redhat.com>
Date:   Wed Dec 9 14:50:41 2015 +0530

    heal : Do not print heal count on ENOTCONN
    
    When a brick is not reachable due to ENOTCONN,
    there are no entries gathered and this information
    should not be printed.
    It is a bug intorduced due to commit
    9c378026e9561595586a817fee0b439e2c863a22, fixing it.
    
    Change-Id: I45559a9560c297854ea6b4177f86e0be30dc6b78
    BUG: 1250803
    Signed-off-by: Anuradha Talur <atalur@redhat.com>
    Reviewed-on: http://review.gluster.org/12919
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 36 Mike McCune 2016-03-28 19:18:47 EDT
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
Comment 37 Niels de Vos 2016-06-16 09:28:57 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.