Bug 1324604
| Summary: | [Perf] : 14-53% regression in metadata performance with RHGS 3.1.3 on FUSE mounts | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Ambarish <asoman> |
| Component: | replicate | Assignee: | Ashish Pandey <aspandey> |
| Status: | CLOSED ERRATA | QA Contact: | Ambarish <asoman> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | rhgs-3.1 | CC: | asoman, aspandey, asrivast, pkarampu, pprakash, rcyriac, rhinduja, rhs-bugs, storage-qa-internal |
| Target Milestone: | --- | Keywords: | Regression, ZStream |
| Target Release: | RHGS 3.1.3 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | glusterfs-3.7.9-3 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-06-23 05:15:52 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1311817 | ||
****** ls -l ****** 3.1.2 : 14230 files/sec 3.1.3 : 12340 files/sec Regression : -14% With older relases ,we used to see close to 25201.4 files/sec ***** stat ***** 3.1.2 : 5693 files/sec 3.1.3 : 2627 files/sec Regression : -53% ******** setxattr ******** 3.1.2 : 5591.6 files/sec 3.1.3 : 2609 files/sec Regression : -53% ***** chmod ***** 3.1.2 : 4543 files/sec 3.1.3 : 2412 files/sec Regression : -46% ******** getxattr ******** 3.1.2 : 21531.3 files/sec 3.1.3 : 21530 files/sec Regression : 0% ---------> PASS A similar issue was tracked via https://bugzilla.redhat.com/show_bug.cgi?id=1287531. Is it possible to get 3.1.3 version with the patch from glusterfs-3.7.5-13.2.git84d7c27.el6rhs.x86_64.rpm which was provided as a fix for the above BZ? Ugggh! I meant Version number = glusterfs-3.7.9-1.el6rhs.x86_64 Hitting the reported issue with 3.7.9-2 build as well. Ugghh.... Please ignore the last comment. Wrong Bug! Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1240 |
Created attachment 1144309 [details] Console logs with baseline(3.1.2) as well as RHGS 3.1.3 Description of problem: Looks like we have regressed with the latest RHGS build while running metadata operations( ls -l,chmod,setxattr and stat) on FUSE mounts.The flux from baseline(3.1.2 final build) varies from 14 to 53%. getxattr looks OK,though. Version-Release number of selected component (if applicable): glusterfs-3.7.5-19.el6rhs.x86_64 How reproducible: 3/3 Steps to Reproduce: 1. Run metadata FOPS thrice using small file benchmark on 3.1.2 and establish a baseline 2. Upgrade to 3.1.3 and run the same tests thrice 3. Acceptance criteria : Percentage change should be within 10% Actual results: ls -l,chmod,setxattr and stat have regressed more than 10%.Details in comments. Expected results: Regression Threshold is +-10%. Additional info: OS : RHEL 6.7 The volume was "performance tuned" [root@gqas001 rpm]# gluster v info Volume Name: testvol Type: Distributed-Replicate Volume ID: 6410a358-53ff-409e-94ee-566937e9ab2d Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: gqas001.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick0 Brick2: gqas014.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick1 Brick3: gqas015.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick2 Brick4: gqas016.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick3 Options Reconfigured: *client.event-threads: 4* *server.event-threads: 4* *cluster.lookup-optimize: on* server.allow-insecure: on performance.stat-prefetch: off performance.readdir-ahead: on [root@gqas001 rpm]# [root@gqas001 rpm]# Smallfiles benchmark was used in a distributed multithreaded manner. The testbed consisted of 4 servers,4 clients(1X mount per server) on 10GbE network. Console logs attached.