Bug 817785 - The posix xlator does not fill d_type for READDIRP replies
Summary: The posix xlator does not fill d_type for READDIRP replies
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: posix
Version: pre-release
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
Assignee: Vijay Bellur
QA Contact: Vijaykumar Koppad
URL: http://review.gluster.com/3256
Whiteboard:
Depends On:
Blocks: 817967
TreeView+ depends on / blocked
 
Reported: 2012-05-01 09:45 UTC by Niels de Vos
Modified: 2014-08-25 00:49 UTC (History)
3 users (show)

Fixed In Version: glusterfs-3.4.0
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-07-24 17:17:25 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)
Little test program taken from 'man 2 getdents' (1.85 KB, text/plain)
2012-05-03 10:49 UTC, Niels de Vos
no flags Details

Description Niels de Vos 2012-05-01 09:45:12 UTC
Description of problem:
d_type should contain the type of the the dir-entry (man 3 readdir). Currently the d_type is always set to DT_UNKNOWN (0). The POSIX standard readdir() returns a 'struct dirent' on both Linux and NetBSD with the d_type attribute.

Note that the storage/bdb-xlator sets d_type already.

Version-Release number of selected component (if applicable):
v3.3.0qa39

Additional info:
Detected while implementing READDIRP in wireshark. Example:

GlusterFS
    [Program Version: 310]
    [GlusterFS: READDIRP (40)]
    Entries returned: 3
    Errno: 2 (Last READDIRP reply)
    Value Follows: Yes
    Entry Path:.
        Inode: 1
        Offset: 4
        Path length: 1
        Type: DT_UNKNOWN (0)
        Path: .
            length: 1
            contents: .
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 1
            ia_dev: 0x000000000000fd02
            mode: 040755
            ia_nlink: 2
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 19
            ia_blksize: 4096
            ia_blocks: 0
            ia_time: 1325950454
            ia_atime_nsec: 246733905
            ia_mtime: 1325950451
            ia_mtime_msec: 644202391
            ia_ctime: 1325950451
            ia_ctime_nsec: 644202391
    Value Follows: Yes
    Entry Path:..
        Inode: 18446744073709551615
        Offset: 6
        Path length: 2
        Type: DT_UNKNOWN (0)
        Path: ..
            length: 2
            contents: ..
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 18446744073709551615
            ia_dev: 0x000000000000fd00
            mode: 040755
            ia_nlink: 3
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 4096
            ia_blksize: 4096
            ia_blocks: 8
            ia_time: 1322567207
            ia_atime_nsec: 85477217
            ia_mtime: 1325949727
            ia_mtime_msec: 735204886
            ia_ctime: 1325949727
            ia_ctime_nsec: 735204886
    Value Follows: Yes
    Entry Path:README
        Inode: 18446744070110182954
        Offset: 512
        Path length: 6
        Type: DT_UNKNOWN (0)
        Path: README
            length: 6
            contents: README
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 18446744070110182954
            ia_dev: 0x000000000000fd02
            mode: 0100644
            ia_nlink: 1
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 36
            ia_blksize: 4096
            ia_blocks: 8
            ia_time: 1325950469
            ia_atime_nsec: 775217282
            ia_mtime: 1325950451
            ia_mtime_msec: 646201892
            ia_ctime: 1325950451
            ia_ctime_nsec: 646201892
    Value Follows: No

Comment 1 Niels de Vos 2012-05-01 09:47:46 UTC
review request: http://review.gluster.com/3256

Comment 2 Niels de Vos 2012-05-03 09:13:06 UTC
Example output with patch 2 from the review (Type now contains DT_*):

GlusterFS
    [Program Version: 310]
    [GlusterFS: READDIRP (40)]
    Entries returned: 6
    Errno: 2 (Last READDIRP reply)
    Value Follows: Yes
    Entry Path:README
        Inode: 18446744071187256986
        Offset: 830133171
        Path length: 6
        Type: DT_REG (8)
        Path: README
            length: 6
            contents: README
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 18446744071187256986
            ia_dev: 0x000000000000fd00
            mode: 0100644
            ia_nlink: 1
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 0
            ia_blksize: 4096
            ia_blocks: 0
            ia_time: 1336034496
            ia_atime_nsec: 0
            ia_mtime: 1336034496
            ia_mtime_msec: 0
            ia_ctime: 1336034496
            ia_ctime_nsec: 336528580
    Value Follows: Yes
    Entry Path:.
        Inode: 1
        Offset: 961315903
        Path length: 1
        Type: DT_DIR (4)
        Path: .
            length: 1
            contents: .
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 1
            ia_dev: 0x000000000000fd00
            mode: 040755
            ia_nlink: 3
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 4096
            ia_blksize: 4096
            ia_blocks: 16
            ia_time: 1336034509
            ia_atime_nsec: 226384831
            ia_mtime: 1336034503
            ia_mtime_msec: 641081262
            ia_ctime: 1336034503
            ia_ctime_nsec: 641081262
    Value Follows: Yes
    Entry Path:subdir.d
        Inode: 18446744072115547470
        Offset: 1340999430
        Path length: 8
        Type: DT_DIR (4)
        Path: subdir.d
            length: 8
            contents: subdir.d
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 18446744072115547470
            ia_dev: 0x000000000000fd00
            mode: 040755
            ia_nlink: 2
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 4096
            ia_blksize: 4096
            ia_blocks: 8
            ia_time: 1336034500
            ia_atime_nsec: 993661184
            ia_mtime: 1336034500
            ia_mtime_msec: 993661184
            ia_ctime: 1336034500
            ia_ctime_nsec: 999236522
    Value Follows: Yes
    Entry Path:in-out.sock
        Inode: 128708907
        Offset: 1468041477
        Path length: 11
        Type: DT_SOCK (12)
        Path: in-out.sock
            length: 11
            contents: in-out.sock
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 128708907
            ia_dev: 0x000000000000fd00
            mode: 0140755
            ia_nlink: 1
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 0
            ia_blksize: 4096
            ia_blocks: 0
            ia_time: 1336034503
            ia_atime_nsec: 641081262
            ia_mtime: 1336034503
            ia_mtime_msec: 641081262
            ia_ctime: 1336034503
            ia_ctime_nsec: 641081262
    Value Follows: Yes
    Entry Path:setgid-test.sh
        Inode: 18446744070032133479
        Offset: 1962579926
        Path length: 14
        Type: DT_REG (8)
        Path: setgid-test.sh
            length: 14
            contents: setgid-test.sh
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 18446744070032133479
            ia_dev: 0x000000000000fd00
            mode: 0100775
            ia_nlink: 1
            ia_uid: 0
            ia_gid: 16777236
            ia_rdev: 0x0000000000000000
            ia_size: 105
            ia_blksize: 4096
            ia_blocks: 16
            ia_time: 1334668308
            ia_atime_nsec: 0
            ia_mtime: 1334668308
            ia_mtime_msec: 0
            ia_ctime: 1334668310
            ia_ctime_nsec: 286311361
    Value Follows: Yes
    Entry Path:..
        Inode: 18446744073709551615
        Offset: 2147483647
        Path length: 2
        Type: DT_DIR (4)
        Path: ..
            length: 2
            contents: ..
            fill bytes: opaque data
        Stat
            GFID: <DATA>
                contents: <DATA>
            ia_ino: 18446744073709551615
            ia_dev: 0x000000000000fd00
            mode: 040755
            ia_nlink: 8
            ia_uid: 0
            ia_gid: 0
            ia_rdev: 0x0000000000000000
            ia_size: 4096
            ia_blksize: 4096
            ia_blocks: 8
            ia_time: 1329560992
            ia_atime_nsec: 397857250
            ia_mtime: 1334317623
            ia_mtime_msec: 518744589
            ia_ctime: 1334317623
            ia_ctime_nsec: 518744589
    Value Follows: No

Comment 3 Niels de Vos 2012-05-03 10:49:51 UTC
Created attachment 581826 [details]
Little test program taken from 'man 2 getdents'

Results over a FUSE mount (big inodes replaced by X):

[root@node1 tmp]# mount -t glusterfs node1:/replicated /mnt/replicated/
[root@node1 tmp]# /tmp/check-dirent-d_type /mnt/replicated/
--------------- nread=184 ---------------
i-node#  file type  d_reclen  d_off   d_name
       X  ???          32  830133171  README
       1  ???          24  961315903  .
       X  ???          32 1340999430  subdir.d
       X  ???          32 1468041477  in-out.sock
       X  ???          40 1962579926  setgid-test.sh
      -1  ???          24 2147483647  ..


Results over a NFS mount:

[root@node1 tmp]# mount -t nfs -o vers=3 node1:/replicated /mnt/dist-repl/
[root@node1 tmp]# /tmp/check-dirent-d_type /mnt/dist-repl/
--------------- nread=184 ---------------
i-node#  file type  d_reclen  d_off   d_name
       X  regular      32          1  README
       1  directory    24          2  .
       X  directory    32          3  subdir.d
       X  socket       32          4  in-out.sock
       X  regular      40          5  setgid-test.sh
       1  directory    24          6  ..


Looking into this further, it seems that NFS gathers the d_type even without patch 2. The NFS-server seems to use xlators/nfs/server/src/nfs3-helpers.c:nfs3_stat_to_fattr3() to fill the d_type and other attributes.


Testes with patch 3 (updated http://review.gluster.com/3256) shows the correct results over FUSE as well:

[root@node1 ~]# mount -t glusterfs node1:/replicated /mnt/replicated/
[root@node1 ~]# /tmp/check-dirent-d_type /mnt/replicated/
--------------- nread=184 ---------------
i-node#  file type  d_reclen  d_off   d_name
       X  regular      32  830133171  README
       1  directory    24  961315903  .
       X  directory    32 1340999430  subdir.d
       X  socket       32 1468041477  in-out.sock
       X  regular      40 1962579926  setgid-test.sh
      -1  directory    24 2147483647  ..

Comment 4 Anand Avati 2012-05-19 03:33:08 UTC
CHANGE: http://review.gluster.com/3256 (storage/posix,fuse-bridge: fill the d_type attribute in READDIRP replies) merged in master by Anand Avati (avati)

Comment 5 Amar Tumballi 2012-05-31 08:11:21 UTC
checked by having a gdb brakepoint in fuse_readdir_cbk, and checking the entries manually. d_type is set to different values now.


Note You need to log in before you can comment on or make changes to this bug.