Bug 1336766

Summary: [Perf] : ~17% regression on FUSE mounts for smallfile mkdirs.
Product: Red Hat Gluster Storage Reporter: Ambarish <asoman>
Component: coreAssignee: Raghavendra G <rgowdapp>
Status: CLOSED WONTFIX QA Contact: Ambarish <asoman>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhgs-3.1CC: amukherj, asoman, ksandha, nbalacha, rcyriac, rgowdapp, rhs-bugs
Target Milestone: ---Keywords: Performance, Regression, ZStream
Target Release: ---Flags: rgowdapp: needinfo? (asoman)
rgowdapp: needinfo? (ksandha)
Hardware: x86_64   
OS: Linux   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-16 18:17:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1369312    
Bug Blocks:    

Description Ambarish 2016-05-17 12:19:05 UTC
Description of problem:

A small regression was introduced with 3.7.9-4 on small file mkdirs.

Benchmark with 3.1.2 : 2716.356667 files/sec

What I'm seeing on 3.7.9-4 is ~2200-2300 files/sec

This is without any performance tuning/enhancement.

More on workload details / volume profiles later.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:

1. Establish baseline for mkdir on 3.1.2. 

2. Run the same test on 3.1.3

3. Check for any regression.Should be within +-10%.

Actual results:

The operation has regressed by ~17%

Expected results:

The results should be within 10% of the baseline.

Additional info:

*OS* : RHEL 6.X

[root@gqas013 ~]# gluster v info testvol
Volume Name: testvol
Type: Distributed-Replicate
Volume ID: 2930ec4a-5f3a-4ac2-a3fc-5b792ef29383
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Brick1: gqas013.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick5
Brick2: gqas011.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick1
Brick3: gqas005.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick2
Brick4: gqas006.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick3
Options Reconfigured:
server.allow-insecure: on
performance.stat-prefetch: off
performance.readdir-ahead: on
[root@gqas013 ~]# 

> rmdir regression is tracked via https://bugzilla.redhat.com/show_bug.cgi?id=1330235

Comment 5 Atin Mukherjee 2016-05-18 09:21:54 UTC
Since the dev_freeze criteria is met, meeting the blocker criteria requires at least 20% drop in performance. So this doesn't qualify as a blocker. Can we have an agreement on that and based on it clear the blocker flag. However dev will initiate an analysis to see what has caused this performance drop. Assigning it to Raghavendra.

Comment 15 Raghavendra G 2017-09-04 09:17:41 UTC

Though I am skeptical that [1] improves performance of mkdir, it would be helpful if we can test it out once a downstream build with [1] is available. Based on the data available we can decide the course of action for this bz.

Putting NEEDINFO on Karan/Ambarish for perf data.