Description of problem: ======================== Perf tests were run on a 6node cluster, having 12 bricks each, hosting a 24*3 volume 'testvol'. The volume was mounted over fuse, and I/O was run from 6 clients on 5000 files of size 64Kb each, via 8 threads. All the readings noted for data and metadata OPs were compared with the last GAed build RHGS 3.4 Batch Update 4. While all of them are in the acceptable discrepancy limit, rename OP is showing a drop of 10%. It is expected for some deviation to be present, and hence a number is arrived after taking an average of multiple runs. Version-Release number of selected component (if applicable): ============================================================ glusterfs-6.0-3 How reproducible: ================= Always Steps to Reproduce: =================== 1. Create a 24*3 volume on a 6node cluster having the build 2. Run small file script from client, over fuse 3. Compare the files/sec reading with the baseline of RHGS 3.4 Batch Update 4 Actual results: =============== Rename shows a drop of 10% Expected results: =================== No drop >5% should be seen in any of the data/metadata OPs Additional info: ================= Please find the comparison below for all the OPs over multiple builds. The same and more readings can be viewed @https://docs.google.com/spreadsheets/d/1r-peEBfHjds27ISwuikWgPJ4djAGQqqZu397wGrWtE4/edit?usp=sharing Sosreports, smallfile script used, perf run logs, volume profile info can be found at: http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/swetas/<bugnumber> Release RHGS 3.4 BU4 RHGS 3.5 - Cluster1 ----------------------------------------------------------- Build "glusterfs-3.12.2-47" glusterfs-6.0-1 glusterfs-6.0-2 glusterfs-6.0-3 3.5 v/s 3.4.4 ------------------------------------------------------------------------------------------------------------------------------------------ Creates 2535 2645.25 2623.25 2619.75 3.34% ls-l 7548 8822.5 8635 8933.13 18.35% chmod 1759 1765.5 1727.25 1766.75 0.44% stat 3163.5 3108.75 3079.25 3113.63 -1.58% Reads 3391.2 3366.4 3324.6 3381.2 -0.29% Appends 1382.25 1366.75 1344.25 1362.13 -1.46% Rename 256.75 227.75 239.25 230 -10.42% Delete-Renamed 2541.2 2601.6 2569.6 2639.2 3.86% Mkdir 743.4 748.2 747.6 762.1 2.52% Rmdir 474.8 464.8 469.4 471.2 -0.76% [root@gqas010 ~]# gluster v info Volume Name: testvol Type: Distributed-Replicate Volume ID: 70cf29b4-08ed-48cf-8ef2-35834f841be9 Status: Started Snapshot Count: 0 Number of Bricks: 24 x 3 = 72 Transport-type: tcp Bricks: Brick1: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol Brick2: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol Brick3: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol Brick4: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol Brick5: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol Brick6: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol Brick7: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol Brick8: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol Brick9: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol Brick10: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol Brick11: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol Brick12: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol Brick13: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol Brick14: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol Brick15: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol Brick16: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol Brick17: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol Brick18: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol Brick19: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol Brick20: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol Brick21: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol Brick22: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol Brick23: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol Brick24: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol Brick25: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol Brick26: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol Brick27: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol Brick28: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol Brick29: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol Brick30: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol Brick31: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol Brick32: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol Brick33: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol Brick34: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol Brick35: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol Brick36: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol Brick37: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol Brick38: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol Brick39: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol Brick40: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol Brick41: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol Brick42: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol Brick43: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol Brick44: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol Brick45: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol Brick46: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol Brick47: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol Brick48: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol Brick49: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol Brick50: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol Brick51: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol Brick52: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol Brick53: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol Brick54: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol Brick55: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol Brick56: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol Brick57: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol Brick58: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol Brick59: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol Brick60: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol Brick61: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol Brick62: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol Brick63: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol Brick64: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol Brick65: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol Brick66: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol Brick67: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol Brick68: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol Brick69: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol Brick70: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol Brick71: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol Brick72: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol Options Reconfigured: server.event-threads: 4 client.event-threads: 4 cluster.lookup-optimize: on network.inode-lru-limit: 200000 performance.md-cache-timeout: 600 performance.cache-invalidation: on performance.stat-prefetch: on features.cache-invalidation-timeout: 600 features.cache-invalidation: on transport.address-family: inet nfs.disable: off performance.client-io-threads: off [root@gqas010 ~]#
Marking this bug as Verified based on Comment 14
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:3249