Description of problem: A poor performance can be observed when copy/create a lot of small files to an EC volume. From profile I saw READDIRP has a large Avg-latency which is 30-40%. Version-Release number of selected component (if applicable): gluster-nagios-addons-0.2.8-1.el7rhgs.x86_64 gluster-nagios-common-0.2.4-1.el7rhgs.noarch glusterfs-3.8.4-18.el7rhgs.x86_64 glusterfs-api-3.8.4-18.el7rhgs.x86_64 glusterfs-cli-3.8.4-18.el7rhgs.x86_64 glusterfs-client-xlators-3.8.4-18.el7rhgs.x86_64 glusterfs-fuse-3.8.4-18.el7rhgs.x86_64 glusterfs-ganesha-3.8.4-18.el7rhgs.x86_64 glusterfs-geo-replication-3.8.4-18.el7rhgs.x86_64 glusterfs-libs-3.8.4-18.el7rhgs.x86_64 glusterfs-rdma-3.8.4-18.el7rhgs.x86_64 glusterfs-server-3.8.4-18.el7rhgs.x86_64 nfs-ganesha-gluster-2.4.1-9.el7rhgs.x86_64 python-gluster-3.8.4-18.el7rhgs.noarch samba-vfs-glusterfs-4.4.6-5.el7rhgs.x86_64 vdsm-gluster-4.17.33-1.1.el7rhgs.noarch How reproducible: 1. every time Steps to Reproduce: 1. Create a 4+2 EC volume on 3 gluster nodes. 2. mount the volume by FUSE and export the mount point to samba 3. access the volume from windows client, and copy 60000 files(7G totally ) to the volume by xcopy command. Actual results: will cost 40+ minutes Expected results: want it to get faster. Additional info: # gluster volume info dispersed1 Volume Name: dispersed1 Type: Disperse Volume ID: 2af122a6-1549-4b38-90ab-06dbae284e45 Status: Started Snapshot Count: 0 Number of Bricks: 1 x (4 + 2) = 6 Transport-type: tcp Bricks: Brick1: 10.0.0.71:/gluster/brick1 Brick2: 10.0.0.71:/gluster/brick2 Brick3: 10.0.0.72:/gluster/brick1 Brick4: 10.0.0.72:/gluster/brick2 Brick5: 10.0.0.73:/gluster/brick1 Brick6: 10.0.0.73:/gluster/brick2 Options Reconfigured: performance.read-ahead: off client.event-threads: 4 performance.client-io-threads: on cluster.lookup-optimize: on cluster.readdir-optimize: on network.inode-lru-limit: 100000 performance.md-cache-timeout: 600 performance.cache-invalidation: on features.cache-invalidation-timeout: 600 features.cache-invalidation: on storage.batch-fsync-delay-usec: 0 performance.cache-samba-metadata: on server.allow-insecure: on performance.stat-prefetch: on performance.readdir-ahead: off nfs.disable: on [wenshi@collab-shell x-text]$ cat vol_profile_0614-3.txt Brick: 10.0.0.71:/gluster/brick2 -------------------------------- Cumulative Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1469 985 4610 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 710 577 727 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 242 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54807 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117077 RELEASEDIR 0.00 209.00 us 209.00 us 209.00 us 1 RENAME 0.00 71.00 us 43.00 us 127.00 us 6 OPEN 0.04 393.07 us 145.00 us 1104.00 us 288 READDIR 0.04 56.57 us 16.00 us 257.00 us 2452 STATFS 0.23 76.24 us 20.00 us 164.00 us 9319 READ 0.77 40.53 us 7.00 us 135.00 us 60090 FLUSH 1.30 2293.09 us 123.00 us 926327.00 us 1776 MKDIR 1.60 91.02 us 21.00 us 606.00 us 55304 XATTROP 1.73 104.05 us 36.00 us 462.00 us 52177 FTRUNCATE 2.07 61.75 us 15.00 us 239.00 us 105165 FSTAT 2.35 89.41 us 23.00 us 232.00 us 82738 SETATTR 2.46 114.54 us 27.00 us 582.00 us 67530 WRITE 3.22 45.21 us 8.00 us 1578.00 us 223709 INODELK 3.32 89.15 us 1.00 us 2131.00 us 117077 OPENDIR 3.92 232.73 us 100.00 us 163513.00 us 52982 CREATE 5.32 102.06 us 9.00 us 655677.00 us 163884 FXATTROP 5.45 205.62 us 14.00 us 386240.00 us 83347 GETXATTR 7.19 136.04 us 31.00 us 1222.00 us 166050 SETXATTR 20.52 100.37 us 11.00 us 1251401.00 us 642623 LOOKUP 38.49 3647.02 us 18.00 us 71724.00 us 33180 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 45759488 bytes Data Written: 2036296192 bytes Interval 0 Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1469 985 4610 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 710 577 727 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 242 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54807 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117077 RELEASEDIR 0.00 209.00 us 209.00 us 209.00 us 1 RENAME 0.00 71.00 us 43.00 us 127.00 us 6 OPEN 0.04 393.07 us 145.00 us 1104.00 us 288 READDIR 0.04 56.57 us 16.00 us 257.00 us 2452 STATFS 0.23 76.24 us 20.00 us 164.00 us 9319 READ 0.77 40.53 us 7.00 us 135.00 us 60090 FLUSH 1.30 2293.09 us 123.00 us 926327.00 us 1776 MKDIR 1.60 91.02 us 21.00 us 606.00 us 55304 XATTROP 1.73 104.05 us 36.00 us 462.00 us 52177 FTRUNCATE 2.07 61.75 us 15.00 us 239.00 us 105165 FSTAT 2.35 89.41 us 23.00 us 232.00 us 82738 SETATTR 2.46 114.54 us 27.00 us 582.00 us 67530 WRITE 3.22 45.21 us 8.00 us 1578.00 us 223709 INODELK 3.32 89.15 us 1.00 us 2131.00 us 117077 OPENDIR 3.92 232.73 us 100.00 us 163513.00 us 52982 CREATE 5.32 102.06 us 9.00 us 655677.00 us 163884 FXATTROP 5.45 205.62 us 14.00 us 386240.00 us 83347 GETXATTR 7.19 136.04 us 31.00 us 1222.00 us 166050 SETXATTR 20.52 100.37 us 11.00 us 1251401.00 us 642623 LOOKUP 38.49 3647.02 us 18.00 us 71724.00 us 33180 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 45759488 bytes Data Written: 2036296192 bytes Brick: 10.0.0.71:/gluster/brick1 -------------------------------- Cumulative Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1484 976 4554 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 728 536 701 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 238 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54807 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117078 RELEASEDIR 0.00 164.00 us 164.00 us 164.00 us 1 RENAME 0.00 75.67 us 50.00 us 104.00 us 6 OPEN 0.04 408.52 us 144.00 us 829.00 us 290 READDIR 0.05 55.47 us 17.00 us 249.00 us 2452 STATFS 0.25 76.35 us 21.00 us 158.00 us 9217 READ 0.50 781.25 us 126.00 us 826464.00 us 1776 MKDIR 0.90 41.71 us 7.00 us 167.00 us 60090 FLUSH 1.86 93.58 us 22.00 us 669.00 us 55304 XATTROP 2.01 107.30 us 40.00 us 250.00 us 52177 FTRUNCATE 2.43 64.21 us 16.00 us 234.00 us 105165 FSTAT 2.79 93.63 us 21.00 us 626.00 us 82738 SETATTR 2.85 117.08 us 28.00 us 4808.00 us 67530 WRITE 3.78 89.75 us 1.00 us 2253.00 us 117078 OPENDIR 3.90 48.41 us 8.00 us 1802.00 us 223708 INODELK 5.01 262.64 us 102.00 us 1368286.00 us 52982 CREATE 5.65 95.74 us 10.00 us 502047.00 us 163885 FXATTROP 6.02 201.00 us 14.00 us 345298.00 us 83217 GETXATTR 8.29 138.66 us 33.00 us 1078.00 us 166050 SETXATTR 23.68 102.38 us 12.00 us 390815.00 us 642625 LOOKUP 30.00 3805.81 us 19.00 us 616226.00 us 21902 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 44471296 bytes Data Written: 2036296192 bytes Interval 0 Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1484 976 4554 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 728 536 701 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 238 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54807 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117078 RELEASEDIR 0.00 164.00 us 164.00 us 164.00 us 1 RENAME 0.00 75.67 us 50.00 us 104.00 us 6 OPEN 0.04 408.52 us 144.00 us 829.00 us 290 READDIR 0.05 55.47 us 17.00 us 249.00 us 2452 STATFS 0.25 76.35 us 21.00 us 158.00 us 9217 READ 0.50 781.25 us 126.00 us 826464.00 us 1776 MKDIR 0.90 41.71 us 7.00 us 167.00 us 60090 FLUSH 1.86 93.58 us 22.00 us 669.00 us 55304 XATTROP 2.01 107.30 us 40.00 us 250.00 us 52177 FTRUNCATE 2.43 64.21 us 16.00 us 234.00 us 105165 FSTAT 2.79 93.63 us 21.00 us 626.00 us 82738 SETATTR 2.85 117.08 us 28.00 us 4808.00 us 67530 WRITE 3.78 89.75 us 1.00 us 2253.00 us 117078 OPENDIR 3.90 48.41 us 8.00 us 1802.00 us 223708 INODELK 5.01 262.64 us 102.00 us 1368286.00 us 52982 CREATE 5.65 95.74 us 10.00 us 502047.00 us 163885 FXATTROP 6.02 201.00 us 14.00 us 345298.00 us 83217 GETXATTR 8.29 138.66 us 33.00 us 1078.00 us 166050 SETXATTR 23.68 102.38 us 12.00 us 390815.00 us 642625 LOOKUP 30.00 3805.81 us 19.00 us 616226.00 us 21902 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 44471296 bytes Data Written: 2036296192 bytes Brick: 10.0.0.72:/gluster/brick1 -------------------------------- Cumulative Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1480 976 4648 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 704 599 747 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 251 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54810 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117080 RELEASEDIR 0.00 188.00 us 188.00 us 188.00 us 1 RENAME 0.00 92.67 us 74.00 us 143.00 us 6 OPEN 0.03 399.89 us 145.00 us 851.00 us 294 READDIR 0.04 58.29 us 16.00 us 154.00 us 2452 STATFS 0.21 76.92 us 22.00 us 168.00 us 9404 READ 0.35 667.04 us 130.00 us 327543.00 us 1776 MKDIR 0.76 42.30 us 8.00 us 157.00 us 60090 FLUSH 1.51 91.79 us 19.00 us 333.00 us 55304 XATTROP 1.66 107.19 us 38.00 us 318.00 us 52177 FTRUNCATE 2.05 65.53 us 16.00 us 346.00 us 105165 FSTAT 2.31 93.82 us 22.00 us 317.00 us 82738 SETATTR 2.39 119.04 us 26.00 us 1006.00 us 67530 WRITE 3.21 92.13 us 1.00 us 1733.00 us 117080 OPENDIR 3.28 49.34 us 9.00 us 1315.00 us 223710 INODELK 4.20 266.79 us 102.00 us 680697.00 us 52982 CREATE 4.90 198.24 us 14.00 us 379080.00 us 83089 GETXATTR 4.93 101.29 us 9.00 us 1004809.00 us 163885 FXATTROP 6.79 137.65 us 33.00 us 1025.00 us 166050 SETXATTR 19.65 102.88 us 17.00 us 836278.00 us 642627 LOOKUP 41.73 3442.23 us 19.00 us 308537.00 us 40788 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 46831616 bytes Data Written: 2036296192 bytes Interval 0 Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1480 976 4648 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 704 599 747 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 251 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54810 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117080 RELEASEDIR 0.00 188.00 us 188.00 us 188.00 us 1 RENAME 0.00 92.67 us 74.00 us 143.00 us 6 OPEN 0.03 399.89 us 145.00 us 851.00 us 294 READDIR 0.04 58.29 us 16.00 us 154.00 us 2452 STATFS 0.21 76.92 us 22.00 us 168.00 us 9404 READ 0.35 667.04 us 130.00 us 327543.00 us 1776 MKDIR 0.76 42.30 us 8.00 us 157.00 us 60090 FLUSH 1.51 91.79 us 19.00 us 333.00 us 55304 XATTROP 1.66 107.19 us 38.00 us 318.00 us 52177 FTRUNCATE 2.05 65.53 us 16.00 us 346.00 us 105165 FSTAT 2.31 93.82 us 22.00 us 317.00 us 82738 SETATTR 2.39 119.04 us 26.00 us 1006.00 us 67530 WRITE 3.21 92.13 us 1.00 us 1733.00 us 117080 OPENDIR 3.28 49.34 us 9.00 us 1315.00 us 223710 INODELK 4.20 266.79 us 102.00 us 680697.00 us 52982 CREATE 4.90 198.24 us 14.00 us 379080.00 us 83089 GETXATTR 4.93 101.29 us 9.00 us 1004809.00 us 163885 FXATTROP 6.79 137.65 us 33.00 us 1025.00 us 166050 SETXATTR 19.65 102.88 us 17.00 us 836278.00 us 642627 LOOKUP 41.73 3442.23 us 19.00 us 308537.00 us 40788 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 46831616 bytes Data Written: 2036296192 bytes Brick: 10.0.0.72:/gluster/brick2 -------------------------------- Cumulative Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1396 889 4500 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 693 599 745 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 229 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54805 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117079 RELEASEDIR 0.00 177.00 us 177.00 us 177.00 us 1 RENAME 0.00 94.83 us 81.00 us 114.00 us 6 OPEN 0.04 362.67 us 122.00 us 805.00 us 292 READDIR 0.05 52.47 us 15.00 us 155.00 us 2452 STATFS 0.23 73.00 us 18.00 us 526.00 us 9051 READ 0.85 1361.63 us 123.00 us 837548.00 us 1776 MKDIR 0.85 40.53 us 8.00 us 158.00 us 60090 FLUSH 1.73 89.38 us 22.00 us 570.00 us 55304 XATTROP 1.90 103.92 us 38.00 us 365.00 us 52177 FTRUNCATE 2.28 61.97 us 15.00 us 162.00 us 105165 FSTAT 2.60 89.48 us 25.00 us 551.00 us 82738 SETATTR 2.77 116.88 us 27.00 us 1363.00 us 67530 WRITE 3.61 87.98 us 1.00 us 1588.00 us 117079 OPENDIR 3.76 47.91 us 8.00 us 1356.00 us 223708 INODELK 4.53 243.82 us 102.00 us 447399.00 us 52982 CREATE 5.31 92.38 us 9.00 us 487535.00 us 163884 FXATTROP 5.54 190.14 us 12.00 us 524210.00 us 83142 GETXATTR 7.82 134.42 us 35.00 us 705.00 us 166050 SETXATTR 22.01 97.68 us 13.00 us 91750.00 us 642629 LOOKUP 34.13 3531.25 us 19.00 us 626560.00 us 27568 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 45528064 bytes Data Written: 2036296192 bytes Interval 0 Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1396 889 4500 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 693 599 745 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 229 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54805 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117079 RELEASEDIR 0.00 177.00 us 177.00 us 177.00 us 1 RENAME 0.00 94.83 us 81.00 us 114.00 us 6 OPEN 0.04 362.67 us 122.00 us 805.00 us 292 READDIR 0.05 52.47 us 15.00 us 155.00 us 2452 STATFS 0.23 73.00 us 18.00 us 526.00 us 9051 READ 0.85 1361.63 us 123.00 us 837548.00 us 1776 MKDIR 0.85 40.53 us 8.00 us 158.00 us 60090 FLUSH 1.73 89.38 us 22.00 us 570.00 us 55304 XATTROP 1.90 103.92 us 38.00 us 365.00 us 52177 FTRUNCATE 2.28 61.97 us 15.00 us 162.00 us 105165 FSTAT 2.60 89.48 us 25.00 us 551.00 us 82738 SETATTR 2.77 116.88 us 27.00 us 1363.00 us 67530 WRITE 3.61 87.98 us 1.00 us 1588.00 us 117079 OPENDIR 3.76 47.91 us 8.00 us 1356.00 us 223708 INODELK 4.53 243.82 us 102.00 us 447399.00 us 52982 CREATE 5.31 92.38 us 9.00 us 487535.00 us 163884 FXATTROP 5.54 190.14 us 12.00 us 524210.00 us 83142 GETXATTR 7.82 134.42 us 35.00 us 705.00 us 166050 SETXATTR 22.01 97.68 us 13.00 us 91750.00 us 642628 LOOKUP 34.13 3531.25 us 19.00 us 626560.00 us 27568 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 45528064 bytes Data Written: 2036296192 bytes Brick: 10.0.0.73:/gluster/brick1 -------------------------------- Cumulative Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1322 826 4242 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 704 549 696 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 217 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54806 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117078 RELEASEDIR 0.00 267.00 us 267.00 us 267.00 us 1 RENAME 0.00 90.17 us 49.00 us 115.00 us 6 OPEN 0.04 317.28 us 137.00 us 600.00 us 290 READDIR 0.05 43.56 us 18.00 us 129.00 us 2452 STATFS 0.23 62.12 us 20.00 us 125.00 us 8556 READ 0.85 32.56 us 8.00 us 152.00 us 60090 FLUSH 1.06 1378.58 us 127.00 us 635595.00 us 1776 MKDIR 1.76 73.61 us 20.00 us 258.00 us 55304 XATTROP 2.01 89.21 us 40.00 us 218.00 us 52177 FTRUNCATE 2.33 51.32 us 14.00 us 145.00 us 105165 FSTAT 2.68 91.75 us 27.00 us 698.00 us 67530 WRITE 2.74 76.75 us 26.00 us 552.00 us 82738 SETATTR 3.55 36.77 us 9.00 us 1323.00 us 223710 INODELK 3.57 70.59 us 1.00 us 1679.00 us 117078 OPENDIR 4.72 206.06 us 106.00 us 377976.00 us 52982 CREATE 5.19 73.35 us 10.00 us 402215.00 us 163885 FXATTROP 6.18 172.41 us 14.00 us 395059.00 us 83023 GETXATTR 7.77 108.40 us 32.00 us 855.00 us 166050 SETXATTR 23.34 84.07 us 14.00 us 323796.00 us 642625 LOOKUP 31.94 3798.80 us 20.00 us 72906.00 us 19464 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 42773504 bytes Data Written: 2036296192 bytes Interval 0 Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1322 826 4242 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 704 549 696 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 217 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54806 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117078 RELEASEDIR 0.00 267.00 us 267.00 us 267.00 us 1 RENAME 0.00 90.17 us 49.00 us 115.00 us 6 OPEN 0.04 317.28 us 137.00 us 600.00 us 290 READDIR 0.05 43.56 us 18.00 us 129.00 us 2452 STATFS 0.23 62.12 us 20.00 us 125.00 us 8556 READ 0.85 32.56 us 8.00 us 152.00 us 60090 FLUSH 1.06 1378.58 us 127.00 us 635595.00 us 1776 MKDIR 1.76 73.61 us 20.00 us 258.00 us 55304 XATTROP 2.01 89.21 us 40.00 us 218.00 us 52177 FTRUNCATE 2.33 51.32 us 14.00 us 145.00 us 105165 FSTAT 2.68 91.75 us 27.00 us 698.00 us 67530 WRITE 2.74 76.75 us 26.00 us 552.00 us 82738 SETATTR 3.55 36.77 us 9.00 us 1323.00 us 223710 INODELK 3.57 70.59 us 1.00 us 1679.00 us 117078 OPENDIR 4.72 206.06 us 106.00 us 377976.00 us 52982 CREATE 5.19 73.35 us 10.00 us 402215.00 us 163885 FXATTROP 6.18 172.41 us 14.00 us 395059.00 us 83023 GETXATTR 7.77 108.40 us 32.00 us 855.00 us 166050 SETXATTR 23.34 84.07 us 14.00 us 323796.00 us 642625 LOOKUP 31.94 3798.80 us 20.00 us 72906.00 us 19464 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 42773504 bytes Data Written: 2036296192 bytes Brick: 10.0.0.73:/gluster/brick2 -------------------------------- Cumulative Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1421 904 4334 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 733 508 672 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 235 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54810 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117079 RELEASEDIR 0.00 257.00 us 257.00 us 257.00 us 1 RENAME 0.00 88.50 us 56.00 us 126.00 us 6 OPEN 0.04 312.10 us 117.00 us 637.00 us 292 READDIR 0.04 41.65 us 14.00 us 137.00 us 2452 STATFS 0.20 59.91 us 22.00 us 125.00 us 8807 READ 0.64 932.89 us 127.00 us 1089879.00 us 1776 MKDIR 0.70 30.30 us 8.00 us 114.00 us 60090 FLUSH 1.47 68.59 us 23.00 us 255.00 us 55304 XATTROP 1.72 85.47 us 40.00 us 310.00 us 52177 FTRUNCATE 1.98 48.73 us 16.00 us 193.00 us 105165 FSTAT 2.26 86.63 us 29.00 us 788.00 us 67530 WRITE 2.32 72.60 us 26.00 us 201.00 us 82738 SETATTR 3.06 67.62 us 1.00 us 1242.00 us 117079 OPENDIR 3.09 35.78 us 10.00 us 1277.00 us 223709 INODELK 3.94 192.76 us 107.00 us 250262.00 us 52982 CREATE 4.73 74.73 us 10.00 us 608318.00 us 163885 FXATTROP 5.62 175.29 us 12.00 us 905368.00 us 83028 GETXATTR 6.62 103.16 us 35.00 us 792.00 us 166050 SETXATTR 20.10 80.99 us 12.00 us 150047.00 us 642623 LOOKUP 41.46 3573.02 us 20.00 us 65494.00 us 30042 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 42788864 bytes Data Written: 2036296192 bytes Interval 0 Stats: Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 1421 904 4334 No. of Writes: 16535 10979 12741 Block Size: 4096b+ 8192b+ 16384b+ No. of Reads: 733 508 672 No. of Writes: 4942 2889 3024 Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 235 0 0 No. of Writes: 4540 4738 3815 Block Size: 262144b+ No. of Reads: 0 No. of Writes: 3327 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 54810 FORGET 0.00 0.00 us 0.00 us 0.00 us 52988 RELEASE 0.00 0.00 us 0.00 us 0.00 us 117079 RELEASEDIR 0.00 257.00 us 257.00 us 257.00 us 1 RENAME 0.00 88.50 us 56.00 us 126.00 us 6 OPEN 0.04 312.10 us 117.00 us 637.00 us 292 READDIR 0.04 41.65 us 14.00 us 137.00 us 2452 STATFS 0.20 59.91 us 22.00 us 125.00 us 8807 READ 0.64 932.89 us 127.00 us 1089879.00 us 1776 MKDIR 0.70 30.30 us 8.00 us 114.00 us 60090 FLUSH 1.47 68.59 us 23.00 us 255.00 us 55304 XATTROP 1.72 85.47 us 40.00 us 310.00 us 52177 FTRUNCATE 1.98 48.73 us 16.00 us 193.00 us 105165 FSTAT 2.26 86.63 us 29.00 us 788.00 us 67530 WRITE 2.32 72.60 us 26.00 us 201.00 us 82738 SETATTR 3.06 67.62 us 1.00 us 1242.00 us 117079 OPENDIR 3.09 35.78 us 10.00 us 1277.00 us 223709 INODELK 3.94 192.76 us 107.00 us 250262.00 us 52982 CREATE 4.73 74.73 us 10.00 us 608318.00 us 163885 FXATTROP 5.62 175.29 us 12.00 us 905368.00 us 83028 GETXATTR 6.62 103.16 us 35.00 us 792.00 us 166050 SETXATTR 20.10 80.99 us 12.00 us 150047.00 us 642623 LOOKUP 41.46 3573.02 us 20.00 us 65494.00 us 30042 READDIRP 0.00 0.00 us 0.00 us 0.00 us 820 UPCALL 0.00 0.00 us 0.00 us 0.00 us 785 CI_IATT 0.00 0.00 us 0.00 us 0.00 us 85 CI_XATTR 0.00 0.00 us 0.00 us 0.00 us 35 CI_FORGET Duration: 2928 seconds Data Read: 42788864 bytes Data Written: 2036296192 bytes
*** Bug 1458985 has been marked as a duplicate of this bug. ***
Hi Wenhan, We did not see any issue with it RHEl 7.3 when we tried the test on our side. As per the comment #17 also it looks like issue with window 7 + xcopy Ashish