Bug 1559884

Summary: Linkto files visible in mount point
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Poornima G <pgurusid>
Component: readdir-aheadAssignee: Poornima G <pgurusid>
Status: CLOSED ERRATA QA Contact: Sweta Anandpara <sanandpa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhgs-3.4CC: amukherj, nbalacha, pgurusid, rhs-bugs, sanandpa, sheggodu, srmukher
Target Milestone: ---   
Target Release: RHGS 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.12.2-7 Doc Type: Bug Fix
Doc Text:
When the gluster volume had a certain combination of volume options set, some of the files appeared twice in mountpoint after performing a readdir operation. The following combination of volume options caused the error: performance.stat-prefetch off performance.readdir-ahead on performance.parallel-readdir on With this fix, when readdir is issued on the mountpoint, no double entries are seen for a single file.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-04 06:45:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1503137    
Attachments:
Description Flags
rhgs34_logs none

Description Poornima G 2018-03-23 13:45:33 UTC 
Description of problem:

On a distribute volume, if the below set of options are enabled:
cluster.rebal-throttle: aggressive
performance.rda-cache-limit: 300Mb
server.allow-insecure: on
performance.readdir-ahead: on
performance.force-readdirp: on
storage.batch-fsync-delay-usec: 0
cluster.min-free-disk: 120GB
nfs.disable: on
performance.stat-prefetch: off
cluster.weighted-rebalance: on
transport.address-family: inet
cluster.lookup-optimize: off
performance.parallel-readdir: on
auto-delete: disable

The linkto files will be visible on the mount point

https://bugzilla.redhat.com/show_bug.cgi?id=1523258#c69

I created a pure distribute volume with 3 bricks, created and renamed files so there would be linkto files created:


[root@rhgs313-6 ~]# mount -t glusterfs -s 192.168.122.6:/rebal1 /mnt/fuse1
[root@rhgs313-6 ~]# cd /mnt/fuse1

[root@rhgs313-6 fuse1]# l /bricks/brick1/rebal1-*
/bricks/brick1/rebal1-1:
total 4.0K
-rw-r--r--. 2 root root 5 Mar 22 23:20 FILE-2

/bricks/brick1/rebal1-2:
total 4.0K
-rw-r--r--. 2 root root 5 Mar 22 23:20 AJKAJDNAL
---------T. 2 root root 0 Mar 22 23:20 FILE-2

/bricks/brick1/rebal1-3:
total 0
---------T. 2 root root 0 Mar 22 23:21 AJKAJDNAL
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# l /bricks/brick1/rebal1-*
/bricks/brick1/rebal1-1:
total 4.0K
-rw-r--r--. 2 root root 5 Mar 22 23:20 FILE-2

/bricks/brick1/rebal1-2:
total 4.0K
-rw-r--r--. 2 root root 5 Mar 22 23:20 AJKAJDNAL
---------T. 2 root root 0 Mar 22 23:20 FILE-2

/bricks/brick1/rebal1-3:
total 0
---------T. 2 root root 0 Mar 22 23:21 AJKAJDNAL
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# 

With the default options, linkto files are not visible on the mount point:

[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# l
total 1.0K
-rw-r--r--. 1 root root 5 Mar 22 23:20 AJKAJDNAL
-rw-r--r--. 1 root root 5 Mar 22 23:20 FILE-2
[root@rhgs313-6 fuse1]# 
[root@rhgs313-6 fuse1]# 


Set the options enabled on mediaset:

[root@rhgs313-6 fuse1]# gluster v set  rebal1 performance.readdir-ahead on
volume set: success
[root@rhgs313-6 fuse1]# gluster v set  rebal1 performance.rda-cache-limit 300Mb
volume set: success
[root@rhgs313-6 fuse1]# gluster v set  rebal1 performance.force-readdirp on
volume set: success
[root@rhgs313-6 fuse1]# gluster v set  rebal1 performance.stat-prefetch off
volume set: success
[root@rhgs313-6 fuse1]# gluster v set  rebal1 performance.parallel-readdir on
volume set: success
[root@rhgs313-6 fuse1]# 

[root@rhgs313-6 fuse1]# l
total 1.0K
-rw-r--r--. 1 root root 5 Mar 22 23:20 AJKAJDNAL
---------T. 1 root root 0 Mar 22 23:21 AJKAJDNAL
-rw-r--r--. 1 root root 5 Mar 22 23:20 FILE-2
---------T. 1 root root 0 Mar 22 23:20 FILE-2


[root@rhgs313-6 fuse1]#  gluster v set  rebal1 performance.parallel-readdir  off
volume set: success
[root@rhgs313-6 fuse1]# l
total 1.0K
-rw-r--r--. 1 root root 5 Mar 22 23:20 AJKAJDNAL
-rw-r--r--. 1 root root 5 Mar 22 23:20 FILE-2
[root@rhgs313-6 fuse1]#

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 2 Poornima G 2018-03-23 13:47:58 UTC 
Patch merged upstream :  https://review.gluster.org/#/c/19172/
Comment 7 Sweta Anandpara 2018-05-16 09:09:34 UTC 
Created attachment 1437202 [details]
rhgs34_logs
Comment 8 Sweta Anandpara 2018-05-16 09:10:23 UTC 
Reproduced this on a RHGS 3.3.1 setup:

- where the said options were enabled on a plain distribute volume -> files' create-rename done to have T-files present from a fuse mount -> (Mountpoint displayed T-files) -> parallel-readdir disabled -> (Mountpoint did not display T-files) -> parallel-readdir enabled and stat-prefetch enabled -> (Mountpoint did not display T-files)

Tried the exact same steps on the build glusterfs-3.12.2-10.el7rhgs.x86_64.

- where mountpoint never displayed T-files in any of the steps.. irrespective of parallel-readdir enabled/disabled, or stat-prefetch enabled/disabled. 

In other words, no dependency seen between paralled-readdir and stat-prefetch to be enabled together. 

Moving this bug to verified. PFA detailed logs.
Comment 11 errata-xmlrpc 2018-09-04 06:45:40 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607
Comment 12 Red Hat Bugzilla 2023-09-14 04:25:54 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days