Bug 1460898 - Enabling parallel-readdir causes dht linkto files to be visible on the mount,
Enabling parallel-readdir causes dht linkto files to be visible on the mount,
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: unclassified (Show other bugs)
3.11
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: bugs@gluster.org
dht-parallel-readdirp
: Reopened, Triaged
Depends On: 1431908
Blocks: 1435942
  Show dependency treegraph
 
Reported: 2017-06-13 01:11 EDT by Poornima G
Modified: 2017-06-28 14:32 EDT (History)
4 users (show)

See Also:
Fixed In Version: glusterfs-3.11.1
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1431908
Environment:
Last Closed: 2017-06-28 14:32:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Poornima G 2017-06-13 01:11:50 EDT
+++ This bug was initially created as a clone of Bug #1431908 +++

Description of problem:
When parallel readdir is enabled, readdir-ahead is loaded below DHT and does a readdir and fetches entries without any associated dict with it and caches it. Consequently readdirp from dht is served from this cache and the check_is_linkfile() fails in dht_readdirp_cbk() thereby not filtering it.

Version-Release number of selected component (if applicable):
master, 3.10

How reproducible:
Always

Steps to Reproduce:
1. Create a dist volume, fuse mount it, create  a file
2. Rename the file so that it hashes to a different brick, creating a linkto file
3. Enable performance,parallel-readdir
4. Do ls from the mount ,it will show both the file and the T file

Actual results:
As above

Expected results:
Should not display the linkto file.

--- Additional comment from Ravishankar N on 2017-03-13 21:07:39 EDT ---

Reported by a user at http://lists.gluster.org/pipermail/gluster-users/2017-March/030254.html

--- Additional comment from Worker Ant on 2017-03-15 01:39:54 EDT ---

REVIEW: https://review.gluster.org/16902 (dht: The xattrs sent in readdirp should be sent in opendir aswell) posted (#1) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-03-15 03:02:25 EDT ---

REVIEW: https://review.gluster.org/16902 (dht: The xattrs sent in readdirp should be sent in opendir aswell) posted (#2) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-03-28 02:50:04 EDT ---

REVIEW: https://review.gluster.org/16902 (dht: The xattrs sent in readdirp should be sent in opendir aswell) posted (#3) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-04-05 03:29:11 EDT ---

REVIEW: https://review.gluster.org/16902 (dht: The xattrs sent in readdirp should be sent in opendir aswell) posted (#4) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-04-05 06:28:16 EDT ---

REVIEW: https://review.gluster.org/16902 (dht: The xattrs sent in readdirp should be sent in opendir aswell) posted (#5) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-04-06 00:04:06 EDT ---

COMMIT: https://review.gluster.org/16902 committed in master by Raghavendra G (rgowdapp@redhat.com) 
------
commit 0f71338e1d7c0b70f4fe3b19c68612fe730d9de2
Author: Poornima G <pgurusid@redhat.com>
Date:   Wed Mar 15 10:26:28 2017 +0530

    dht: The xattrs sent in readdirp should be sent in opendir aswell
    
    As readdir-ahead can be loaded as a child of dht, dht has to specify
    the xattrs it is intrested in, as part of opendir call itself.
    
    Change-Id: I012ef96cc143b0cef942df78aa7150d85ec38606
    BUG: 1431908
    Signed-off-by: Poornima G <pgurusid@redhat.com>
    Reviewed-on: https://review.gluster.org/16902
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>

--- Additional comment from Worker Ant on 2017-04-13 07:17:01 EDT ---

REVIEW: https://review.gluster.org/17057 (dht: Add missing braces in dht_opendir) posted (#2) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-04-13 11:57:47 EDT ---

COMMIT: https://review.gluster.org/17057 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit af218797fa98f2f75594fc9ae595f184682f1a0d
Author: Poornima G <pgurusid@redhat.com>
Date:   Thu Apr 13 16:44:44 2017 +0530

    dht: Add missing braces in dht_opendir
    
    Change-Id: I6adce98f52e17953f501bc590ff7189cceac3c31
    BUG: 1431908
    Signed-off-by: Poornima G <pgurusid@redhat.com>
    Reviewed-on: https://review.gluster.org/17057
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>

--- Additional comment from Shyamsundar on 2017-05-30 14:47:11 EDT ---

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.11.0, please open a new bug report.

glusterfs-3.11.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-May/000073.html
[2] https://www.gluster.org/pipermail/gluster-users/

--- Additional comment from Worker Ant on 2017-06-12 01:42:37 EDT ---

REVIEW: https://review.gluster.org/17504 (readdir-ahead: Fix duplicate listing and cache size calculation) posted (#1) for review on master by Poornima G (pgurusid@redhat.com)

--- Additional comment from Worker Ant on 2017-06-12 06:18:28 EDT ---

COMMIT: https://review.gluster.org/17504 committed in master by Raghavendra G (rgowdapp@redhat.com) 
------
commit e97c32ee9913969a726f8a8286cf714f907729d6
Author: Poornima G <pgurusid@redhat.com>
Date:   Mon Jun 12 10:59:04 2017 +0530

    readdir-ahead: Fix duplicate listing and cache size calculation
    
    Issue:
    If a opendir is followed by a closedir without readdir, though
    the prefetched entries were freed, the freed size was not accounted
    in priv->rda_cache_size. Thus the cache limit will exceed if there
    are multiple opendir followed by closedir.
    
    Fix:
    Fix the pric->rda_cache_size calculation. Also have removed the
    inode_ctx_size. Each perf xlator has its own cache limit that
    it works with. Also the inode_ctx size can change, if a forget/
    invalidate or any other factor triggers the inode_ctx size.
    
    Change-Id: I9707ec558076ce046e58a55989ec9513c70ea029
    BUG: 1431908
    Signed-off-by: Poornima G <pgurusid@redhat.com>
    Reviewed-on: https://review.gluster.org/17504
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Comment 1 Worker Ant 2017-06-13 01:12:48 EDT
REVIEW: https://review.gluster.org/17529 (readdir-ahead: Fix duplicate listing and cache size calculation) posted (#1) for review on release-3.11 by Poornima G (pgurusid@redhat.com)
Comment 2 Worker Ant 2017-06-21 08:48:13 EDT
COMMIT: https://review.gluster.org/17529 committed in release-3.11 by Shyamsundar Ranganathan (srangana@redhat.com) 
------
commit 1f4cc8fb8d5647ef923e6cfa7e4f027d6aab97f8
Author: Poornima G <pgurusid@redhat.com>
Date:   Mon Jun 12 10:59:04 2017 +0530

    readdir-ahead: Fix duplicate listing and cache size calculation
    
    Issue:
    If a opendir is followed by a closedir without readdir, though
    the prefetched entries were freed, the freed size was not accounted
    in priv->rda_cache_size. Thus the cache limit will exceed if there
    are multiple opendir followed by closedir.
    
    Fix:
    Fix the pric->rda_cache_size calculation. Also have removed the
    inode_ctx_size. Each perf xlator has its own cache limit that
    it works with. Also the inode_ctx size can change, if a forget/
    invalidate or any other factor triggers the inode_ctx size.
    
    
    > Reviewed-on: https://review.gluster.org/17504
    > Smoke: Gluster Build System <jenkins@build.gluster.org>
    > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    > CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    > Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
    > (cherry picked from commit e97c32ee9913969a726f8a8286cf714f907729d6)
    
    Change-Id: I9707ec558076ce046e58a55989ec9513c70ea029
    BUG: 1460898
    Signed-off-by: Poornima G <pgurusid@redhat.com>
    Reviewed-on: https://review.gluster.org/17529
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Comment 3 Shyamsundar 2017-06-28 14:32:55 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.11.1, please open a new bug report.

glusterfs-3.11.1 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-June/000074.html
[2] https://www.gluster.org/pipermail/gluster-users/

Note You need to log in before you can comment on or make changes to this bug.