Bug 782291

Summary: [glusterfs-3.2.6qa1]: metadata modification operations very slow on nfs when quota/gsync is enabled
Product: [Community] GlusterFS Reporter: Rahul C S <rahulcs>
Component: nfsAssignee: Vivek Agarwal <vagarwal>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: high    
Version: pre-releaseCC: gluster-bugs, rfortier, sankarshan
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.4.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-07-24 17:51:08 UTC Type: ---
Regression: --- Mount Type: nfs
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Performance result none

Description Rahul C S 2012-01-17 05:44:33 UTC 
Description of problem:
[root@Engg test_deploy]# ./calc_avg 4 5 6 7 57 58 59 60

run4 - nfs - 3.2.5qa4 - distrep - (quota off, gsync off)
run5 - nfs - 3.2.5qa4 - distrep - (quota off, gsync on)
run6 - nfs - 3.2.5qa4 - distrep - (quota on, gsync on)
run7 - nfs - 3.2.5qa4 - distrep - (quota on, gsync off)
run57 - nfs - 3.2.6qa1 - distrep - (quota off, gsync off)
run58 - nfs - 3.2.6qa1 - distrep - (quota off, gsync on)
run59 - nfs - 3.2.6qa1 - distrep - (quota on, gsync on)
run60 - nfs - 3.2.6qa1 - distrep - (quota on, gsync off)

Operations                      RUN04   RUN05   RUN06   RUN07   RUN57   RUN58   RUN59   RUN60
-------------------------       ------- ------- ------- ------- ------- ------- ------- -------
emptyfiles_create               227     230     227     242     255     247     260     256
emptyfiles_delete               119     120     123     134     131     125     132     135
smallfiles_create               3018    3129    2989    2923    3118    3207    3120    3208
smallfiles_rewrite              3016    3145    3006    2930    3107    3207    3114    3221
smallfiles_read                 313     310     294     313     328     329     330     316
smallfiles_reread               212     225     227     232     231     237     246     240
smallfiles_delete               137     136     136     140     142     141     147     146
largefile_create                20      19      20      20      20      20      19      20
largefile_rewrite               20      19      20      20      20      20      19      20
largefile_read                  9       9       9       9       9       9       9       9
largefile_reread                0       0       0       0       0       0       0       0
largefile_delete                0       0       0       0       0       0       0       0
directory_crawl_create          275     284     280     303     306     302     320     320
directory_crawl                 12      13      13      14      14      14      15      15
directory_recrawl               89      92      49      50      96      975     54      53
metadata_modify                 235     257     274     298     261     827     1018    1289
directory_crawl_delete          149     148     153     173     170     154     157     167


The runs were generated from our performance testing. As we can see, during metadata modification operations when either quota/geosync is enabled or when both are enabled together, its slower by about 4 times.
Comment 1 Rahul C S 2012-02-20 08:50:41 UTC 
Though there seems to be little improvement on NFS, the metadata modify operations are still slower in 3.2.6qa3 as compared to 3.2.5qa4

I have attached the result sheet.
Comment 2 Rahul C S 2012-02-20 08:51:15 UTC 
Created attachment 564338 [details]
Performance result
Comment 4 Krishna Srinivas 2012-08-14 11:47:28 UTC 
Can this be reproduced and can I take a look at the setup? this was before we removed performance translators from the nfs stack so need to see if we still hit this bug.