Bug 1020029

Summary: oo-reclassify doesn't reclassify threads
Product: OpenShift Online Reporter: Andy Grimm <agrimm>
Component: ContainersAssignee: mfisher
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: bmeng, jgoulding, xtian, yadu
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-24 03:24:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Andy Grimm 2013-10-16 20:13:44 UTC
The code used to decide which processes to reclassify into cgroups only looks at top-level processes (/proc/[0-9]*) rather than all tasks (/proc/[0-9]*/tasks/*).

On a related note, the check in oo-accept-node is similarly incorrect.

Comment 1 Andy Grimm 2013-10-16 20:32:42 UTC
That should have been /proc/[0-9]*/task/* (not "tasks")

Comment 2 Rob Millner 2013-10-16 22:04:42 UTC
Pull request:
https://github.com/openshift/origin-server/pull/3910


Here's how to verify for Q/E...

1. Create a non-scaled jbosseap app and note the gear uuid
  - both spawn a ton of threads

2. Find out how many threads are in the gear cgroup, should be over 60.

wc -l /cgroup/all/openshift/[UUID]/tasks


3. Move the tasks to the root cgroup

cat /cgroup/all/openshift/[UUID]/tasks | while read t; echo $t > /cgroup/all/tasks; done

4. This should now be 0, no threads in the gear cgroup

wc -l /cgroup/all/openshift/[UUID]/tasks

5. Repair the cgroup

oo-cgroup-reclassify -c [UUID]

6. This should now be the number from #2 (or very close)

wc -l /cgroup/all/openshift/[UUID]/tasks

Comment 3 openshift-github-bot 2013-10-17 02:19:13 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/b65344195881663880fb52302b7060cd67ccd3e5
Bug 1020029 - needed to enumerate all threads in the task dir.

Comment 5 Meng Bo 2013-10-17 06:47:15 UTC
[root@ip-10-238-172-200 ~]# cat /cgroup/all/openshift/525f86d3617446e3e4000074/tasks |wc -l
61
[root@ip-10-238-172-200 ~]# cat /cgroup/all/openshift/525f86d3617446e3e4000074/tasks | while read t ;do echo $t > /cgroup/all/tasks ;done
[root@ip-10-238-172-200 ~]# cat /cgroup/all/openshift/525f86d3617446e3e4000074/tasks |wc -l
0
[root@ip-10-238-172-200 ~]# oo-cgroup-reclassify -c 525f86d3617446e3e4000074
[root@ip-10-238-172-200 ~]# cat /cgroup/all/openshift/525f86d3617446e3e4000074/tasks |wc -l
64
[root@ip-10-238-172-200 ~]# 


Checked on devenv_3907, issue has been fixed.

Move to verified.

Comment 6 Andy Grimm 2013-10-17 14:53:47 UTC
Moving back to VERIFIED, since this wasn't actually in yesterday's release.