Bug 1479335 - [GSS]glusterfsd is reaching 1200% CPU utilization [NEEDINFO]
Summary: [GSS]glusterfsd is reaching 1200% CPU utilization
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: replicate
Version: rhgs-3.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: RHGS 3.4.0
Assignee: Mohit Agrawal
QA Contact: Vijay Avuthu
Depends On: 1484446
Blocks: 1503135
TreeView+ depends on / blocked
Reported: 2017-08-08 11:39 UTC by Abhishek Kumar
Modified: 2018-09-17 11:29 UTC (History)
11 users (show)

Fixed In Version: glusterfs-3.12.2-2
Doc Type: Bug Fix
Doc Text:
Some gluster daemons like glustershd have a higher cpu or memory consumption, when there is a large amount of data/entries to healed. This results in slow consumption of resources. You can resolve this by running the control-cpu-load.sh script. This script used the control groups for regulating cpu and memory of any gluster daemon.
Clone Of:
Last Closed: 2018-09-04 06:34:23 UTC
Target Upstream Version:
srmukher: needinfo? (moagrawa)

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2607 None None None 2018-09-04 06:35:54 UTC

Description Abhishek Kumar 2017-08-08 11:39:27 UTC
Description of problem:

glusterfsd is reaching 1200% CPU utilization

Version-Release number of selected component (if applicable):

glusterfs-3.8.4-18.6.el7rhgs.x86_64 (Upgraded Node)
glusterfs-3.8.4-18.4.el7rhgs.x86_64 (Non-upgraded node)

How reproducible:

Customer Environment

Steps to Reproduce:

Gluster cluster is replica 3 with 2 data nodes + arbiter node. Customer has upgraded arbiter node to RHEL7.4 with 'yum update' command. They have rebooted the node and everything was working fine. They repeated the same with another data node but when node comes up after upgrade, CPU utilization got spiked and volume become in-accessible (Work load is a RHEV environment where VMs are consuming this volume becomes non-responding.).

Actual results:

glusterfsd is reaching 1200% CPU utilization

Expected results:

Glusterfsd processes shouldn't got spiked to the amount where volume become in-accessible

Additional info:

Gluster volume is being used in RHEV environment as a back-end storage for VMs.

Comment 12 Ravishankar N 2017-09-27 09:44:25 UTC
Mohit's patch upstream: https://review.gluster.org/#/c/18404/

Comment 15 Vijay Avuthu 2018-05-17 05:03:24 UTC
This bug has been verified as part of bug 1478395

Changing status to Verified.

Comment 16 Srijita Mukherjee 2018-09-03 15:32:57 UTC
Have updated the doc text. kindly review and confirm.

Comment 18 errata-xmlrpc 2018-09-04 06:34:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.