Bug 1705681

Summary: Append regressing by 12% in replica 3 volume over NFS v4 protocol
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Sweta Anandpara <sanandpa>
Component: coreAssignee: Amar Tumballi <atumball>
Status: CLOSED NOTABUG QA Contact: Sweta Anandpara <sanandpa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rhgs-3.5CC: rhs-bugs, sankarshan, storage-qa-internal
Target Milestone: ---Keywords: Performance, Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-05-03 08:29:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sweta Anandpara 2019-05-02 18:24:25 UTC
Description of problem:
=======================
Over multiple runs done in NFSv4.0 protocol, a lot of inconsistency was noticed in results of varios file ops and metadata ops. An average was taken over 3 runs and 'append' op was found to regress by about 12%.

Will continue subsequent runs to check if there is a positive trend noticed.. Will keep this bug updated with results. 

Version-Release number of selected component (if applicable):
============================================================
glusterfs-6.0-2

How reproducible:
=================
Inconsistent results. Average results have been taken over 3 runs.

Steps to Reproduce:
===================
1. On a 6 node cluster with 12 bricks each, create a 24*3 volume 'testvol'
2. Run small file tool for creates, read, append, mkdir, rmdir. These ops were chosen as they figured in border range of ~10% in the previous runs.
3. Compare the numbers with baseline of RHGS 3.4 Update 4

Actual results:
===============
Drop of performance seen in append FOP to about 12%

Expected results:
==================
Regression (drop) if seen should not be >10%

Additional info:
================
[root@gqas001 ~]# gluster volume info
 
Volume Name: gluster_shared_storage
Type: Replicate
Volume ID: 64262ff1-eb76-4381-a05f-defa181618b3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: gqas004.sbu.lab.eng.bos.redhat.com:/var/lib/glusterd/ss_brick
Brick2: gqas014.sbu.lab.eng.bos.redhat.com:/var/lib/glusterd/ss_brick
Brick3: gqas001.sbu.lab.eng.bos.redhat.com:/var/lib/glusterd/ss_brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
nfs-ganesha: enable
cluster.enable-shared-storage: enable
 
Volume Name: testvol
Type: Distributed-Replicate
Volume ID: fc0a47b0-138b-4913-8685-fcb1cd305475
Status: Started
Snapshot Count: 0
Number of Bricks: 24 x 3 = 72
Transport-type: tcp
Bricks:
Brick1: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol
Brick2: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol
Brick3: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol
Brick4: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol
Brick5: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol
Brick6: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick1/testvol
Brick7: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol
Brick8: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol
Brick9: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol
Brick10: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol
Brick11: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol
Brick12: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick2/testvol
Brick13: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol
Brick14: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol
Brick15: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol
Brick16: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol
Brick17: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol
Brick18: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick3/testvol
Brick19: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol
Brick20: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol
Brick21: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol
Brick22: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol
Brick23: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol
Brick24: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick4/testvol
Brick25: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol
Brick26: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol
Brick27: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol
Brick28: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol
Brick29: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol
Brick30: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick5/testvol
Brick31: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol
Brick32: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol
Brick33: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol
Brick34: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol
Brick35: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol
Brick36: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick6/testvol
Brick37: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol
Brick38: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol
Brick39: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol
Brick40: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol
Brick41: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol
Brick42: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick7/testvol
Brick43: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol
Brick44: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol
Brick45: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol
Brick46: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol
Brick47: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol
Brick48: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick8/testvol
Brick49: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol
Brick50: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol
Brick51: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol
Brick52: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol
Brick53: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol
Brick54: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick9/testvol
Brick55: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol
Brick56: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol
Brick57: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol
Brick58: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol
Brick59: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol
Brick60: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick10/testvol
Brick61: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol
Brick62: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol
Brick63: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol
Brick64: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol
Brick65: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol
Brick66: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick11/testvol
Brick67: gqas001.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol
Brick68: gqas004.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol
Brick69: gqas009.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol
Brick70: gqas010.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol
Brick71: gqas012.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol
Brick72: gqas014.sbu.lab.eng.bos.redhat.com:/gluster/brick12/testvol
Options Reconfigured:
ganesha.enable: on
server.event-threads: 4
client.event-threads: 4
cluster.lookup-optimize: on
network.inode-lru-limit: 200000
performance.md-cache-timeout: 600
performance.cache-invalidation: on
performance.stat-prefetch: on
features.cache-invalidation-timeout: 600
features.cache-invalidation: on
transport.address-family: inet
nfs.disable: off
performance.client-io-threads: off
nfs-ganesha: enable
cluster.enable-shared-storage: enable
[root@gqas001 ~]# gluster pool list
UUID					Hostname                          	State
83705b3e-b1c6-4d64-bee0-1e55425b263b	gqas004.sbu.lab.eng.bos.redhat.com	Connected 
b24bb0c4-3c60-42af-bff4-473b48b6b294	gqas014.sbu.lab.eng.bos.redhat.com	Connected 
dab8813c-250b-4858-b57a-4bcb403a6ead	gqas010.sbu.lab.eng.bos.redhat.com	Connected 
1a743b4d-ee15-4462-bdea-f1ea2746d4ab	gqas012.sbu.lab.eng.bos.redhat.com	Connected 
fbfde667-b292-4ede-a0ff-58841c1432a0	gqas009.sbu.lab.eng.bos.redhat.com	Connected 
1673bf7e-d3fd-440b-a844-8128f0deb500	localhost                         	Connected 
[root@gqas001 ~]# 
[root@gqas001 ~]# gluster v get all all
Option                                  Value                                   
------                                  -----                                   
cluster.server-quorum-ratio             51                                      
cluster.enable-shared-storage           enable                                  
cluster.op-version                      70000                                   
cluster.max-op-version                  70000                                   
cluster.brick-multiplex                 disable                                 
cluster.max-bricks-per-process          250                                     
glusterd.vol_count_per_thread           100                                     
cluster.daemon-log-level                INFO                                    
[root@gqas001 ~]# 
[root@gqas001 ~]# 
[root@gqas001 ~]# gluster v status
Status of volume: gluster_shared_storage
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick gqas004.sbu.lab.eng.bos.redhat.com:/v
ar/lib/glusterd/ss_brick                    49152     0          Y       23705
Brick gqas014.sbu.lab.eng.bos.redhat.com:/v
ar/lib/glusterd/ss_brick                    49152     0          Y       24030
Brick gqas001.sbu.lab.eng.bos.redhat.com:/v
ar/lib/glusterd/ss_brick                    49152     0          Y       26815
Self-heal Daemon on localhost               N/A       N/A        Y       26836
Self-heal Daemon on gqas010.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23412
Self-heal Daemon on gqas004.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23726
Self-heal Daemon on gqas014.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       24051
Self-heal Daemon on gqas012.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23727
Self-heal Daemon on gqas009.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23759
 
Task Status of Volume gluster_shared_storage
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: testvol
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick1/testvol                       49153     0          Y       2860 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick1/testvol                       49153     0          Y       21914
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick1/testvol                       49152     0          Y       21902
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick1/testvol                       49152     0          Y       21546
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick1/testvol                       49152     0          Y       21863
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick1/testvol                       49153     0          Y       22262
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick2/testvol                       49154     0          Y       2880 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick2/testvol                       49154     0          Y       21934
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick2/testvol                       49153     0          Y       21922
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick2/testvol                       49153     0          Y       21566
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick2/testvol                       49153     0          Y       21883
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick2/testvol                       49154     0          Y       22282
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick3/testvol                       49155     0          Y       2900 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick3/testvol                       49155     0          Y       21954
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick3/testvol                       49154     0          Y       21943
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick3/testvol                       49154     0          Y       21587
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick3/testvol                       49154     0          Y       21904
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick3/testvol                       49155     0          Y       22303
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick4/testvol                       49156     0          Y       2920 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick4/testvol                       49156     0          Y       21974
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick4/testvol                       49155     0          Y       21963
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick4/testvol                       49155     0          Y       21607
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick4/testvol                       49155     0          Y       21924
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick4/testvol                       49156     0          Y       22323
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick5/testvol                       49157     0          Y       2940 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick5/testvol                       49157     0          Y       21994
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick5/testvol                       49156     0          Y       21983
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick5/testvol                       49156     0          Y       21627
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick5/testvol                       49156     0          Y       21944
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick5/testvol                       49157     0          Y       22343
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick6/testvol                       49158     0          Y       2960 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick6/testvol                       49158     0          Y       22014
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick6/testvol                       49157     0          Y       22003
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick6/testvol                       49157     0          Y       21647
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick6/testvol                       49157     0          Y       21964
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick6/testvol                       49158     0          Y       22363
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick7/testvol                       49159     0          Y       2980 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick7/testvol                       49159     0          Y       22034
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick7/testvol                       49158     0          Y       22023
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick7/testvol                       49158     0          Y       21667
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick7/testvol                       49158     0          Y       21984
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick7/testvol                       49159     0          Y       22383
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick8/testvol                       49160     0          Y       3000 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick8/testvol                       49160     0          Y       22054
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick8/testvol                       49159     0          Y       22043
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick8/testvol                       49159     0          Y       21687
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick8/testvol                       49159     0          Y       22004
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick8/testvol                       49160     0          Y       22403
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick9/testvol                       49161     0          Y       3020 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick9/testvol                       49161     0          Y       22074
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick9/testvol                       49160     0          Y       22063
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick9/testvol                       49160     0          Y       21707
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick9/testvol                       49160     0          Y       22024
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick9/testvol                       49161     0          Y       22423
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick10/testvol                      49162     0          Y       3040 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick10/testvol                      49162     0          Y       22094
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick10/testvol                      49161     0          Y       22083
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick10/testvol                      49161     0          Y       21727
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick10/testvol                      49161     0          Y       22044
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick10/testvol                      49162     0          Y       22443
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick11/testvol                      49163     0          Y       3060 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick11/testvol                      49163     0          Y       22114
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick11/testvol                      49162     0          Y       22103
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick11/testvol                      49162     0          Y       21747
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick11/testvol                      49162     0          Y       22064
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick11/testvol                      49163     0          Y       22463
Brick gqas001.sbu.lab.eng.bos.redhat.com:/g
luster/brick12/testvol                      49164     0          Y       3080 
Brick gqas004.sbu.lab.eng.bos.redhat.com:/g
luster/brick12/testvol                      49164     0          Y       22134
Brick gqas009.sbu.lab.eng.bos.redhat.com:/g
luster/brick12/testvol                      49163     0          Y       22123
Brick gqas010.sbu.lab.eng.bos.redhat.com:/g
luster/brick12/testvol                      49163     0          Y       21767
Brick gqas012.sbu.lab.eng.bos.redhat.com:/g
luster/brick12/testvol                      49163     0          Y       22084
Brick gqas014.sbu.lab.eng.bos.redhat.com:/g
luster/brick12/testvol                      49164     0          Y       22483
Self-heal Daemon on localhost               N/A       N/A        Y       26836
NFS Server on localhost                     N/A       N/A        N       N/A  
Self-heal Daemon on gqas014.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       24051
NFS Server on gqas014.sbu.lab.eng.bos.redha
t.com                                       N/A       N/A        N       N/A  
Self-heal Daemon on gqas004.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23726
NFS Server on gqas004.sbu.lab.eng.bos.redha
t.com                                       N/A       N/A        N       N/A  
Self-heal Daemon on gqas010.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23412
NFS Server on gqas010.sbu.lab.eng.bos.redha
t.com                                       N/A       N/A        N       N/A  
Self-heal Daemon on gqas012.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23727
NFS Server on gqas012.sbu.lab.eng.bos.redha
t.com                                       N/A       N/A        N       N/A  
Self-heal Daemon on gqas009.sbu.lab.eng.bos
.redhat.com                                 N/A       N/A        Y       23759
NFS Server on gqas009.sbu.lab.eng.bos.redha
t.com                                       N/A       N/A        N       N/A  
 
Task Status of Volume testvol
------------------------------------------------------------------------------
There are no active volume tasks
 
[root@gqas001 ~]# 
[root@gqas001 ~]# rpm -qa | grep gluster
glusterfs-client-xlators-6.0-2.el7rhgs.x86_64
tmp-rhs-tests-beaker-rhs-gluster-qe-libs-dev-bturner-3.0-0.noarch
python2-gluster-6.0-2.el7rhgs.x86_64
glusterfs-libs-6.0-2.el7rhgs.x86_64
glusterfs-api-6.0-2.el7rhgs.x86_64
glusterfs-geo-replication-6.0-2.el7rhgs.x86_64
glusterfs-6.0-2.el7rhgs.x86_64
glusterfs-cli-6.0-2.el7rhgs.x86_64
nfs-ganesha-gluster-2.5.5-12.el7rhgs.x86_64
glusterfs-server-6.0-2.el7rhgs.x86_64
glusterfs-ganesha-6.0-2.el7rhgs.x86_64
glusterfs-fuse-6.0-2.el7rhgs.x86_64
[root@gqas001 ~]#