Bug 1404113
| Summary: | [Perf] 12% Drop in sequential reads on SMB v1.0 on a *2 deployment | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Karan Sandha <ksandha> |
| Component: | io-threads | Assignee: | Ravishankar N <ravishankar> |
| Status: | CLOSED DEFERRED | QA Contact: | Nag Pavan Chilakam <nchilaka> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rhgs-3.2 | CC: | amukherj, gdeschner, ksandha, ravishankar, rhs-bugs, rhs-smb |
| Target Milestone: | --- | Keywords: | Regression, ZStream |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-11-09 03:54:53 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
How does that relate to Bug #1397854 ? Is this still an issue? If not can we close this bug? I don't think we would be working on fixing rhgs-3.2 any more.Closing the BZ for now. Please re-open if needed or perf issues are seen on the latest rhgs version. |
Description of problem: There's a 12% regression on sequential reads on SMB v1.0 3.1.3 :- 667526 KB/s 3.8.4.5 :- 587465 KB/s Version-Release number of selected component (if applicable): 3.8.4.5 How reproducible: 100% Steps to Reproduce: 1. Create 4 servers and 4 clients 2. Mount the clients on SMB v1.0 3. Run the iozone command :- #qeSyncDropCaches.sh #iozone -+m /clients.ioz -+h $(hostname) -C -w -c -e -i 1 -+n -r 64k -s 8g -t 16 4. Compare the numbers with 3.1.3 Actual results: There is a regression on 12% on sequential reads Expected results: The shouldn't be any regression. Additional info: [root@gqas003 ~]# gluster v info Volume Name: testvol Type: Distributed-Replicate Volume ID: c16f6e36-0e41-4a35-9966-51ef44d8fc29 Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: gqas003.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick0 Brick2: gqas004.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick1 Brick3: gqas007.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick2 Brick4: gqas012.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick3 Options Reconfigured: server.allow-insecure: on performance.stat-prefetch: off performance.readdir-ahead: on Volume profile for 3.8.4.5 for SMB v1 reads [root@gqas003 ~]# gluster volume profile testvol info Brick: gqas003.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick0 ---------------------------------------------------------------- Cumulative Stats: Block Size: 131072b+ No. of Reads: 470658 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 146.00 us 129.00 us 156.00 us 4 GETXATTR 0.01 96.29 us 57.00 us 157.00 us 38 FSTAT 0.03 204.38 us 185.00 us 387.00 us 72 LOOKUP 99.96 180.90 us 71.00 us 41153.00 us 233461 READ Duration: 565 seconds Data Read: 61690085376 bytes Data Written: 0 bytes Interval 0 Stats: Block Size: 131072b+ No. of Reads: 470658 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 146.00 us 129.00 us 156.00 us 4 GETXATTR 0.01 96.29 us 57.00 us 157.00 us 38 FSTAT 0.03 204.38 us 185.00 us 387.00 us 72 LOOKUP 99.96 180.90 us 71.00 us 41153.00 us 233461 READ Duration: 565 seconds Data Read: 61690085376 bytes Data Written: 0 bytes Brick: gqas007.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick2 ---------------------------------------------------------------- Cumulative Stats: Block Size: 131072b+ No. of Reads: 627420 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 149.60 us 131.00 us 163.00 us 5 GETXATTR 0.01 92.42 us 55.00 us 139.00 us 43 FSTAT 0.02 208.16 us 190.00 us 294.00 us 56 LOOKUP 99.97 178.00 us 71.00 us 64396.00 us 311581 READ Duration: 565 seconds Data Read: 82237194240 bytes Data Written: 0 bytes Interval 0 Stats: Block Size: 131072b+ No. of Reads: 627420 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 149.60 us 131.00 us 163.00 us 5 GETXATTR 0.01 92.42 us 55.00 us 139.00 us 43 FSTAT 0.02 208.16 us 190.00 us 294.00 us 56 LOOKUP 99.97 178.01 us 71.00 us 64396.00 us 311582 READ Duration: 565 seconds Data Read: 82237194240 bytes Data Written: 0 bytes Brick: gqas012.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick3 ---------------------------------------------------------------- Cumulative Stats: Block Size: 131072b+ No. of Reads: 234180 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 146.50 us 136.00 us 157.00 us 2 GETXATTR 0.01 97.17 us 58.00 us 139.00 us 18 FSTAT 0.05 188.45 us 150.00 us 339.00 us 56 LOOKUP 99.94 186.35 us 72.00 us 19972.00 us 114990 READ Duration: 565 seconds Data Read: 30694440960 bytes Data Written: 0 bytes Interval 0 Stats: Block Size: 131072b+ No. of Reads: 234180 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 146.50 us 136.00 us 157.00 us 2 GETXATTR 0.01 97.17 us 58.00 us 139.00 us 18 FSTAT 0.05 188.45 us 150.00 us 339.00 us 56 LOOKUP 99.94 186.35 us 72.00 us 19972.00 us 114990 READ Duration: 565 seconds Data Read: 30694440960 bytes Data Written: 0 bytes Brick: gqas004.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick1 ---------------------------------------------------------------- Cumulative Stats: Block Size: 131072b+ No. of Reads: 533214 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 150.60 us 138.00 us 192.00 us 5 GETXATTR 0.01 94.38 us 57.00 us 132.00 us 50 FSTAT 0.03 204.81 us 133.00 us 311.00 us 72 LOOKUP 99.96 178.41 us 61.00 us 34804.00 us 262891 READ Duration: 565 seconds Data Read: 69889425408 bytes Data Written: 0 bytes Interval 0 Stats: Block Size: 131072b+ No. of Reads: 533214 No. of Writes: 0 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 3 RELEASEDIR 0.00 150.60 us 138.00 us 192.00 us 5 GETXATTR 0.01 94.38 us 57.00 us 132.00 us 50 FSTAT 0.03 204.81 us 133.00 us 311.00 us 72 LOOKUP 99.96 178.41 us 61.00 us 34804.00 us 262891 READ Duration: 565 seconds Data Read: 69889425408 bytes Data Written: 0 bytes Volume profile for 3.1.3 smb v1 reads:- [root@gqas003 ~]# gluster volume profile testvol info Brick: gqas003.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick0 ---------------------------------------------------------------- Cumulative Stats: Block Size: 65536b+ 131072b+ No. of Reads: 0 1012239 No. of Writes: 100748 539446 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 29 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.00 3.00 us 3.00 us 3.00 us 2 OPENDIR 0.00 131.25 us 21.00 us 191.00 us 12 GETXATTR 0.00 739.25 us 298.00 us 1263.00 us 4 READDIR 0.01 79.98 us 51.00 us 118.00 us 61 FSTAT 0.02 190.55 us 22.00 us 491.00 us 74 LOOKUP 99.97 168.24 us 68.00 us 21108.00 us 503315 READ Duration: 34902 seconds Data Read: 132676190208 bytes Data Written: 77309411328 bytes Interval 0 Stats: Block Size: 65536b+ 131072b+ No. of Reads: 0 1012239 No. of Writes: 100748 539446 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 29 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.00 3.00 us 3.00 us 3.00 us 2 OPENDIR 0.00 131.25 us 21.00 us 191.00 us 12 GETXATTR 0.00 739.25 us 298.00 us 1263.00 us 4 READDIR 0.01 79.98 us 51.00 us 118.00 us 61 FSTAT 0.02 190.55 us 22.00 us 491.00 us 74 LOOKUP 99.97 168.24 us 68.00 us 21108.00 us 503315 READ Duration: 34902 seconds Data Read: 132676190208 bytes Data Written: 77309411328 bytes Brick: gqas007.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick2 ---------------------------------------------------------------- Cumulative Stats: Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 0 0 288381 No. of Writes: 1 82543 417478 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 26 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.00 3.50 us 3.00 us 4.00 us 2 OPENDIR 0.00 119.67 us 18.00 us 167.00 us 6 GETXATTR 0.01 98.00 us 51.00 us 135.00 us 17 FSTAT 0.01 751.00 us 347.00 us 1234.00 us 4 READDIR 0.04 186.31 us 34.00 us 280.00 us 58 LOOKUP 99.94 178.51 us 69.00 us 18705.00 us 142597 READ Duration: 34902 seconds Data Read: 37798674432 bytes Data Written: 60129542144 bytes Interval 0 Stats: Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 0 0 288381 No. of Writes: 1 82543 417478 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 26 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.00 3.50 us 3.00 us 4.00 us 2 OPENDIR 0.00 119.67 us 18.00 us 167.00 us 6 GETXATTR 0.01 98.00 us 51.00 us 135.00 us 17 FSTAT 0.01 751.00 us 347.00 us 1234.00 us 4 READDIR 0.04 186.31 us 34.00 us 280.00 us 58 LOOKUP 99.94 178.51 us 69.00 us 18705.00 us 142597 READ Duration: 34902 seconds Data Read: 37798674432 bytes Data Written: 60129542144 bytes Brick: gqas004.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick1 ---------------------------------------------------------------- Cumulative Stats: Block Size: 65536b+ 131072b+ No. of Reads: 0 0 No. of Writes: 100748 539446 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 29 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.04 3.00 us 3.00 us 3.00 us 2 OPENDIR 1.97 108.00 us 89.00 us 140.00 us 3 GETXATTR 18.38 754.25 us 308.00 us 1410.00 us 4 READDIR 79.61 176.64 us 89.00 us 249.00 us 74 LOOKUP Duration: 34902 seconds Data Read: 0 bytes Data Written: 77309411328 bytes Interval 0 Stats: Block Size: 65536b+ 131072b+ No. of Reads: 0 0 No. of Writes: 100748 539446 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 29 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.04 3.00 us 3.00 us 3.00 us 2 OPENDIR 1.97 108.00 us 89.00 us 140.00 us 3 GETXATTR 18.38 754.25 us 308.00 us 1410.00 us 4 READDIR 79.61 176.64 us 89.00 us 249.00 us 74 LOOKUP Duration: 34902 seconds Data Read: 0 bytes Data Written: 77309411328 bytes Brick: gqas012.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick3 ---------------------------------------------------------------- Cumulative Stats: Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 0 0 377236 No. of Writes: 1 82543 417478 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 26 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.00 3.50 us 3.00 us 4.00 us 2 OPENDIR 0.00 130.86 us 83.00 us 158.00 us 7 GETXATTR 0.01 103.42 us 66.00 us 146.00 us 24 FSTAT 0.01 746.00 us 361.00 us 1261.00 us 4 READDIR 0.03 191.74 us 83.00 us 330.00 us 58 LOOKUP 99.95 182.41 us 69.00 us 16281.00 us 186660 READ Duration: 34902 seconds Data Read: 49445076992 bytes Data Written: 60129542144 bytes Interval 0 Stats: Block Size: 32768b+ 65536b+ 131072b+ No. of Reads: 0 0 377236 No. of Writes: 1 82543 417478 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 26 RELEASE 0.00 0.00 us 0.00 us 0.00 us 118 RELEASEDIR 0.00 3.50 us 3.00 us 4.00 us 2 OPENDIR 0.00 130.86 us 83.00 us 158.00 us 7 GETXATTR 0.01 103.42 us 66.00 us 146.00 us 24 FSTAT 0.01 746.00 us 361.00 us 1261.00 us 4 READDIR 0.03 191.74 us 83.00 us 330.00 us 58 LOOKUP 99.95 182.41 us 69.00 us 16281.00 us 186660 READ Duration: 34902 seconds Data Read: 49445076992 bytes Data Written: 60129542144 bytes [root@gqas003 ~]#