Description of problem: ======================= Create a file with dd command on nfs mount and listed the same with ' ls -l'. Its shows as 'T' file most of the times. This works correctly on fuse mount. This is an EC volume as cold tier and 1x2 rep hot tier. [root@rhs-client17 nfs]# dd if=/dev/urandom of=file1 bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 98.7191 s, 10.9 MB/s [root@rhs-client17 nfs]# ls -l file1 -rw-r--r--. 1 root root 1073741824 Dec 8 09:35 file1 [root@rhs-client17 nfs]# ls -l file1 -rw-r--r--. 1 root root 1073741824 Dec 8 09:35 file1 [root@rhs-client17 nfs]# for i in `seq 1 50`; do ls -l ; done total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 [root@rhs-client17 nfs]# ls -l total 2097152 -rw-r--r--. 1 root root 1073741824 Dec 7 14:46 file -rw-r--r--. 1 root root 1073741824 Dec 8 09:35 file1 [root@rhs-client17 nfs]# ls -l total 0 ---------T. 1 root root 0 Dec 7 14:04 file ---------T. 1 root root 0 Dec 8 09:27 file1 [root@rhs-client17 nfs]# ls -l total 2097152 -rw-r--r--. 1 root root 1073741824 Dec 7 14:46 file -rw-r--r--. 1 root root 1073741824 Dec 8 09:35 file1 [root@rhs-client17 nfs]# ls -l total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 [root@rhs-client17 nfs]# ls -l total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 [root@rhs-client17 nfs]# ls -l total 0 ---------T. 1 root root 0 Dec 7 14:03 file ---------T. 1 root root 0 Dec 8 09:27 file1 [root@rhs-client17 nfs]# [root@rhs-client17 nfs]# df . Filesystem 1K-blocks Used Available Use% Mounted on transformers:/disperse_vol1 15971684352 2790400 15968893952 1% /mnt/nfs [root@rhs-client17 nfs]# [root@transformers ~]# gluster v info disperse_vol1 Volume Name: disperse_vol1 Type: Tier Volume ID: a6f27d4d-6838-47bf-ba2d-43fe9c980be3 Status: Started Number of Bricks: 14 Transport-type: tcp Hot Tier : Hot Tier Type : Replicate Number of Bricks: 1 x 2 = 2 Brick1: ninja:/rhs/brick1/vol1-tier2 Brick2: vertigo:/rhs/brick1/vol1-tier1 Cold Tier: Cold Tier Type : Distributed-Disperse Number of Bricks: 2 x (4 + 2) = 12 Brick3: transformers:/rhs/brick1/b1 Brick4: interstellar:/rhs/brick1/b2 Brick5: transformers:/rhs/brick2/b3 Brick6: interstellar:/rhs/brick2/b4 Brick7: transformers:/rhs/brick3/b5 Brick8: interstellar:/rhs/brick3/b6 Brick9: transformers:/rhs/brick4/b7 Brick10: interstellar:/rhs/brick4/b8 Brick11: transformers:/rhs/brick5/b9 Brick12: interstellar:/rhs/brick5/b10 Brick13: transformers:/rhs/brick6/b11 Brick14: interstellar:/rhs/brick6/b12 Options Reconfigured: cluster.watermark-hi: 2 cluster.watermark-low: 1 cluster.tier-mode: cache features.ctr-enabled: on server.event-threads: 2 client.event-threads: 2 features.quota-deem-statfs: on features.inode-quota: on features.quota: on features.uss: on performance.readdir-ahead: on [root@transformers ~]# Version-Release number of selected component (if applicable): ============================================================ 3.7.5-9 How reproducible: ================= 100% Steps to Reproduce: =================== As in description Actual results: =============== Regular files are listed as 'T' files on nfs mount Expected results: Additional info:
This is seen on RHEL7.2 / NFS.
The http://review.gluster.org/#/c/12960/, fixes the above mentioned problem. But it may need to call an additional getattr for those files from the client side. In numbers, for performing readdirp(ls -ltRr ) on dir which contains 10 sub directories , each with 1000 files with 1k size. time taken before applying the patch: real 0m32.070s user 0m0.445s sys 0m2.213s after applying the patch: real 0m34.403s user 0m0.535s sys 0m3.046s
https://code.engineering.redhat.com/gerrit/64016
verified on glusterfs-3.7.5-12 and have not seen the issue. Marking this as fixed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0193.html