Bug 1479335

Summary: [GSS]glusterfsd is reaching 1200% CPU utilization
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Abhishek Kumar <abhishku>
Component: replicateAssignee: Mohit Agrawal <moagrawa>
Status: CLOSED ERRATA QA Contact: Vijay Avuthu <vavuthu>
Severity: high Docs Contact:
Priority: high    
Version: rhgs-3.2CC: abhishku, amukherj, ksubrahm, moagrawa, nchilaka, ravishankar, rhinduja, rhs-bugs, sheggodu, srmukher, storage-qa-internal
Target Milestone: ---   
Target Release: RHGS 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.12.2-2 Doc Type: Bug Fix
Doc Text:
Some gluster daemons like glustershd have a higher cpu or memory consumption, when there is a large amount of data/entries to healed. This results in slow consumption of resources. You can resolve this by running the control-cpu-load.sh script. This script used the control groups for regulating cpu and memory of any gluster daemon.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-04 06:34:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1484446    
Bug Blocks: 1503135    

Description Abhishek Kumar 2017-08-08 11:39:27 UTC
Description of problem:

glusterfsd is reaching 1200% CPU utilization

Version-Release number of selected component (if applicable):

glusterfs-3.8.4-18.6.el7rhgs.x86_64 (Upgraded Node)
glusterfs-3.8.4-18.4.el7rhgs.x86_64 (Non-upgraded node)

How reproducible:

Customer Environment

Steps to Reproduce:

Gluster cluster is replica 3 with 2 data nodes + arbiter node. Customer has upgraded arbiter node to RHEL7.4 with 'yum update' command. They have rebooted the node and everything was working fine. They repeated the same with another data node but when node comes up after upgrade, CPU utilization got spiked and volume become in-accessible (Work load is a RHEV environment where VMs are consuming this volume becomes non-responding.).

Actual results:

glusterfsd is reaching 1200% CPU utilization

Expected results:

Glusterfsd processes shouldn't got spiked to the amount where volume become in-accessible

Additional info:

Gluster volume is being used in RHEV environment as a back-end storage for VMs.

Comment 12 Ravishankar N 2017-09-27 09:44:25 UTC
Mohit's patch upstream: https://review.gluster.org/#/c/18404/

Comment 15 Vijay Avuthu 2018-05-17 05:03:24 UTC
This bug has been verified as part of bug 1478395

Changing status to Verified.

Comment 16 Srijita Mukherjee 2018-09-03 15:32:57 UTC
Have updated the doc text. kindly review and confirm.

Comment 18 errata-xmlrpc 2018-09-04 06:34:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607

Comment 19 Red Hat Bugzilla 2023-09-14 04:02:08 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days