Bug 1284495 - cgrulesengd is not moving all threads to the configured cpuset
cgrulesengd is not moving all threads to the configured cpuset
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libcgroup (Show other bugs)
6.8
x86_64 Linux
unspecified Severity high
: rc
: ---
Assigned To: Nikola Forró
Chao Ye
: Patch
Depends On:
Blocks: 1269194 1356036 1359264
  Show dependency treegraph
 
Reported: 2015-11-23 08:34 EST by Daniel Bristot de Oliveira
Modified: 2017-03-21 05:11 EDT (History)
5 users (show)

See Also:
Fixed In Version: libcgroup-0.40.rc1-22.el6
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-03-21 05:11:12 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Daniel Bristot de Oliveira 2015-11-23 08:34:52 EST
Description of problem:

The cgrulesengd is moving only the thread with the same PID/TID to the
cgroup/cpuset set on /etc/cgrules.conf. That is a problem because
the process is not completely moved to a cpuset, but only the fisrt thread
of the process.

Version-Release number of selected component (if applicable):
libcgroup-0.40.rc1-16.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
On a RHEL6 default install, with libcgroup installed. I am using the dbus-deamon
to show the problem, but it reproduces with any multi-threaded application.

1. File the cgconfig.conf and cgrules.conf with the following content:

>>>> /etc/cgconfig.conf >>>>
mount {
        cpuset = /cgroup/cpuset;
}

group dbus {
        cpuset {
                cpuset.mems="0";
                cpuset.cpus="0";
        }
}
<<<< /etc/cgconfig.conf <<<<

>>>> /etc/cgrules.conf >>>>
dbus:dbus-daemon        cpuset  dbus/
<<<< /etc/cgrules.conf <<<<

2. Enable both services:

  chkconfig cgconfig on
  chkconfig cgred on

  (check if the dbus-deamon is enabled...)

3. Reboot the system.

4. See that only the tread with the same PID/TID were moved to the cpuset. For
example:

	# ps -eLo pid,lwp,user,comm | grep dbus
	 1510  1510 dbus     dbus-daemon
	 1510  1512 dbus     dbus-daemon
	# cd /cgroup/cpuset/dbus/
	# cat tasks 
	1510
	# cat ../tasks | grep 1512
	1512

Actual results:
Only the thread with same PID/TID is moved to the cpuset.

Expected results:
All threads of a process should be moved to the configured cpuset.

Additional info:

Stracing the cgrulesengd (with --verbose --logfile=log --debug -n), it
shows that:


	open("/proc/1510/status", O_RDONLY|O_CLOEXEC) = 4
	fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
	mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd808bbf000
	read(4, "Name:\tdbus-daemon\nState:\tS (slee"..., 1024) = 880
	write(1, "Scanned proc values are 81 81 81"..., 36) = 36
	write(1, "Scanned proc values are 81 81 81"..., 36) = 36
	close(4)                                = 0
	munmap(0x7fd808bbf000, 4096)            = 0
	open("/proc/1510/status", O_RDONLY|O_CLOEXEC) = 4
	fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
	mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd808bbf000
	read(4, "Name:\tdbus-daemon\nState:\tS (slee"..., 1024) = 880
	close(4)                                = 0
	munmap(0x7fd808bbf000, 4096)            = 0
	readlink("/proc/1510/exe", "/bin/dbus-daemon", 4096) = 16
	write(1, "Found matching rule dbus for PID"..., 57) = 57
	write(1, "Executing rule dbus for PID 1510"..., 36) = 36
	write(1, "Will move pid 1510 to cgroup 'db"..., 37) = 37
	write(1, "Adding controller cpuset\n", 25) = 25
	open("/cgroup/cpuset/dbus//tasks", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
	fstat(4, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
	mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd808bbf000
	write(4, "1510", 4)                     = 4
	close(4)                                = 0
	munmap(0x7fd808bbf000, 4096)            = 0
	write(1, "OK!\n", 4)                    = 4

The cgrulesengd reads all /proc/PID/{status|exe} to match the process.
If it matches, it includes the PID on the cpuset's task file.
The problem is that the "task" deals on thread level. So, it only moved the
thread with the TID=PID!

To move all threads to the expected cgroup, it should either:
	Write the PROCESS PID to the cpuset.procs file, or
	Write all THREADS TID to the tasks file.

NOTE: do not use the ps to check thread's cgroup, check BZ1284076
Comment 10 errata-xmlrpc 2017-03-21 05:11:12 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0583.html

Note You need to log in before you can comment on or make changes to this bug.