Bug 1641908 - [Perf] Performance Drop observed in FUSE mount with FIPs enable/disable
Summary: [Perf] Performance Drop observed in FUSE mount with FIPs enable/disable
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: core
Version: rhgs-3.4
Hardware: Unspecified
OS: Linux
low
medium
Target Milestone: ---
: ---
Assignee: Xavi Hernandez
QA Contact: Rahul Hinduja
URL:
Whiteboard:
Depends On: RHGS-Bad-IO
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-10-23 07:33 UTC by Sachin P Mali
Modified: 2019-08-16 04:28 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-16 04:28:24 UTC
Embargoed:


Attachments (Terms of Use)

Description Sachin P Mali 2018-10-23 07:33:11 UTC
Description of problem:

I ran the small-file performance test on FUSE mounted volume. After test, I observed performance drop in Create and Rename FOPs by 16 to 20 %. Also, My test include testing of FIPs enable but here I observe performance drop in 'delete-rename' FOPs , here I observed performance drop by 32%

Details : 
Our small file test does below things,
      1. Create 5000 on local as well as remote client (6 clients)
      2. Do Read, write, append, rename,delete-rename etc operation.

Step to Reproduce : 
     1. Configure 6 node cluster with replica-3 volume type
     2. Mount volume using 'glusterfs' FS type.
     3. Start IO using small-file tool
     4. Calculate result. 

Gluster version : 
###############
[root@gqas001 ~]# rpm -qa | grep gluster
glusterfs-fuse-3.12.2-23.el7rhgs.x86_64
glusterfs-rdma-3.12.2-23.el7rhgs.x86_64
glusterfs-libs-3.12.2-23.el7rhgs.x86_64
glusterfs-3.12.2-23.el7rhgs.x86_64
glusterfs-cli-3.12.2-23.el7rhgs.x86_64
glusterfs-geo-replication-3.12.2-23.el7rhgs.x86_64
tmp-rhs-tests-beaker-rhs-gluster-qe-libs-dev-bturner-3.0-0.noarch
glusterfs-client-xlators-3.12.2-23.el7rhgs.x86_64
glusterfs-debuginfo-3.12.2-18.el7.x86_64
glusterfs-server-3.12.2-23.el7rhgs.x86_64
python2-gluster-3.12.2-23.el7rhgs.x86_64
glusterfs-api-3.12.2-23.el7rhgs.x86_64
[root@gqas001 ~]# 
###############
Options :
###############
performance.cache-samba-metadata: on
server.event-threads: 4
client.event-threads: 4
cluster.lookup-optimize: on
network.inode-lru-limit: 90000
performance.md-cache-timeout: 600
performance.cache-invalidation: on
performance.stat-prefetch: on
features.cache-invalidation-timeout: 600
features.cache-invalidation: on
transport.address-family: inet
nfs.disable: off
performance.client-io-threads: off
################
Volume Type : Replica-3
Cluster : 6 servers and 6 clients

FIPs: Enable : 

################
[root@gqas001 ~]# cat /boot/grub2/grub.cfg | grep -i fip
	linux16 /vmlinuz-3.10.0-957.el7.x86_64 root=/dev/mapper/rhel_gqas001-root ro crashkernel=auto rd.lvm.lv=rhel_gqas001/root rd.lvm.lv=rhel_gqas001/swap selinux=0 console=ttyS1,115200 LANG=en_US.UTF-8 fips=1 boot=UUID=03794366-b295-4b3a-91c4-02c9553bab56
[root@gqas001 ~]# 
################
RHEL version : RHEL 7.6

Test Results : 
https://docs.google.com/spreadsheets/d/1B2obaj3w85O2xtzqc_anyJoDy9Nnejg3jvBl3mWivMc/edit#gid=0

Note : Please let me know, If you require more information on it.

Comment 4 Sachin P Mali 2018-10-24 05:32:12 UTC
I have provided the information in above comment.

Comment 5 Amar Tumballi 2018-10-24 06:23:10 UTC
Do we have any performance number on basic local filesystem when FIPS mode is enabled on RHEL? Ie, if the local filesystem itself is regressing 10% due to fips mode, it is possible to see 20+% with glusterfs.

Comment 6 Sachin P Mali 2018-10-25 06:41:14 UTC
Hi Amar, 
Our testing include only external protocols, (i.e. FUSE,GNFS,SAMBA,NFS-GANESHA). We don't have any performance number of local filesystem.


Note You need to log in before you can comment on or make changes to this bug.