Bug 1764202 - cgroup control-cpu-load.sh script not working
Summary: cgroup control-cpu-load.sh script not working
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: core
Version: rhgs-3.5
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: RHGS 3.5.0
Assignee: Mohit Agrawal
QA Contact: Nag Pavan Chilakam
URL:
Whiteboard:
Depends On:
Blocks: 1696809 1764208 1766424 1766425
TreeView+ depends on / blocked
 
Reported: 2019-10-22 13:04 UTC by Nag Pavan Chilakam
Modified: 2019-10-31 09:22 UTC (History)
5 users (show)

Fixed In Version: glusterfs-6.0-20
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1764208 (view as bug list)
Environment:
Last Closed: 2019-10-30 12:23:03 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2019:3249 0 None None None 2019-10-30 12:23:26 UTC

Description Nag Pavan Chilakam 2019-10-22 13:04:31 UTC
Description of problem:
==============================
i ran cgroup script to control cpu usage as glusterfsd was consuming more than 3000%
However it is not bring the cpu consumption under the mentioned range.

I am referring to script /usr/share/glusterfs/scripts/control-cpu-load.sh


Important note: I didn't test control-mem.sh
Kindly check the same for even control-mem.sh
Version-Release number of selected component (if applicable):
==================
6.0.15

How reproducible:
===========
always


[root@rhs-gp-srv2 ~]# top -n 1 -b|egrep 'RES|gluster'
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 6722 root      20   0   19.9g   3.8g   3428 S  1250  8.1 229868:38 glusterfsd
 7421 root      20   0   13.4g   4.7g   4416 S  30.0  9.9   5002:00 glusterfs
 6173 root      20   0  596100  23596   2548 S   0.0  0.0   0:20.29 glusterd
 7410 root      20   0 1453872  16424   2220 S   0.0  0.0   6:18.91 glusterfsd
[root@rhs-gp-srv2 ~]# 
[root@rhs-gp-srv2 ~]# pwd
/root
[root@rhs-gp-srv2 ~]# cd /usr/share/glusterfs/scripts
[root@rhs-gp-srv2 scripts]# ./control-cpu-load.sh 
Enter gluster daemon pid for which you want to control CPU.
glusterfsd
Entered daemon_pid is not numeric so Rerun the script.
[root@rhs-gp-srv2 scripts]# ./control-cpu-load.sh 
Enter gluster daemon pid for which you want to control CPU.
6722
If you want to continue the script to attach 6722 with new cgroup_gluster_6722 cgroup Press (y/n)?n
no
[root@rhs-gp-srv2 scripts]# ./control-cpu-load.sh 
Enter gluster daemon pid for which you want to control CPU.
6722
If you want to continue the script to attach 6722 with new cgroup_gluster_6722 cgroup Press (y/n)?y
yes
Creating child cgroup directory 'cgroup_gluster_6722 cgroup' for glusterd.service.
Enter quota value in range [10,100]:  
25
Entered quota value is 25
Setting 25000 to cpu.cfs_quota_us for gluster_cgroup.
Tasks are attached successfully specific to 6722 to cgroup_gluster_6722.
[root@rhs-gp-srv2 scripts]# top -n 1 -b|egrep 'RES|gluster'
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 6722 root      20   0   19.9g   3.8g   3392 S  2845  8.1 229895:21 glusterfsd
 7421 root      20   0   13.4g   4.9g   4416 S  30.0 10.5   5002:22 glusterfs
 6173 root      20   0  596100  23448   2400 S   0.0  0.0   0:20.29 glusterd
 7410 root      20   0 1453872  16424   2220 S   0.0  0.0   6:18.91 glusterfsd
[root@rhs-gp-srv2 scripts]# top -n 1 -b|egrep 'RES|gluster'
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 6722 root      20   0   19.9g   3.8g   3392 S  2640  8.1 229896:07 glusterfsd
 7421 root      20   0   13.4g   5.0g   4416 S  25.0 10.6   5002:23 glusterfs
 6173 root      20   0  596100  23448   2400 S   0.0  0.0   0:20.29 glusterd
 7410 root      20   0 1453872  16424   2220 S   0.0  0.0   6:18.91 glusterfsd
[root@rhs-gp-srv2 scripts]# top -n 1 -b|egrep 'RES|gluster'
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 6722 root      20   0   19.9g   3.8g   3392 S  2479  8.1 229896:38 glusterfsd
 7421 root      20   0   13.4g   5.0g   4416 S  31.6 10.6   5002:23 glusterfs
 6173 root      20   0  596100  23448   2400 S   0.0  0.0   0:20.29 glusterd
 7410 root      20   0 1453872  16424   2220 S   0.0  0.0   6:18.91 glusterfsd



Steps to Reproduce:
=================
1.have a setup where cpu consumption of gluster procs are high
2. run /usr/share/glusterfs/scripts/control-cpu-load.sh 
3. enter the pid of gluster proc and the limit under which the cpu must be consumed.

Actual results:
=============
script runs but cpu consumption not coming under control

Comment 9 Nag Pavan Chilakam 2019-10-24 12:40:51 UTC
tested with the script made available with 6.0.20 packages.
The cpu consumption is now under control with the usage of control-cpu-load.sh script
hence moving to verified

Comment 11 errata-xmlrpc 2019-10-30 12:23:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:3249


Note You need to log in before you can comment on or make changes to this bug.