Bug 1307175 - oo-accept-node does not validate whether threads are in cgroups
oo-accept-node does not validate whether threads are in cgroups
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers (Show other bugs)
2.2.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Miciah Dashiel Butler Masters
DeShuai Ma
: UpcomingRelease
: 1308740 (view as bug list)
Depends On: 1067107 1308740
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-12 17:29 EST by Miciah Dashiel Butler Masters
Modified: 2016-03-22 12:54 EDT (History)
8 users (show)

See Also:
Fixed In Version: openshift-origin-node-util-1.38.6.1-1.el6op
Doc Type: Bug Fix
Doc Text:
Cause: Consequence: Fix: Result: The oo-accept-node script was incorrectly verifying that processes were in cgroups procs. Output from this script may have been misleading as the script should have been checking that threads are in cgroup tasks. Oo-accept-node will now correctly compare threads with cgroup tasks.
Story Points: ---
Clone Of: 1067107
Environment:
Last Closed: 2016-03-22 12:54:22 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Miciah Dashiel Butler Masters 2016-02-12 17:29:54 EST
+++ This bug was initially created as a clone of Bug #1067107 +++

Description of problem:

oo-accept-node only verifies that processes are in cgroups, not threads.  It should be using the -L flag to ps, and comparing against tasks instead of cgroup.procs

This was mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1020029 , but was not addressed as part of that fix.

--- Additional comment from openshift-github-bot on 2016-01-28 16:00:26 EST ---

Commits pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/5c6699e31aee4139d976bb9a254d252eebdb719c
oo-accept-node: update to check cgroup tasks

Bug 1067107
BZ https://bugzilla.redhat.com/show_bug.cgi?id=1067107

Updates oo-accept-node to check thread ids against cgroup tasks instead of
checking process ids against cgroup procs.

https://github.com/openshift/origin-server/commit/c9ee5b56766f9cfb0b2b9e35b21b32ba4b30e971
Merge pull request #6347 from thrasher-redhat/bug1067107

Merged by openshift-bot

--- Additional comment from Rory Thrasher on 2016-01-28 17:08:38 EST ---

QA,

Please verify that oo-accept-node now checks threads against tasks and will report when a thread is missing from the gear's tasks.

1. Create a multi-threaded app and then any second app.

# rhc app-create myapp jbossas-7
# rhc app-create other python-3.3

2. Check the cgroup tasks for a resource in 'myapp' and pick a thread id.

# cat /cgroup/cpu/openshift/<myapp_uuid>/tasks
7621
...
7655

3. Add the task id to the same resource in another gear (in 'other'), which will remove it from the original tasks.

# echo 7655 > /cgroup/cpu/openshift/<other_uuid>/tasks

4. Verify that oo-accept-node will find the missing task with an error message similar to "#{uuid} has a thread: tid:#{tid}, pid:#{pid} missing from cgroups controller: #{controller}".

# oo-accept-node


Thank you.
Comment 4 Rory Thrasher 2016-02-22 15:21:14 EST
*** Bug 1308740 has been marked as a duplicate of this bug. ***
Comment 6 Anping Li 2016-02-26 04:11:26 EST
Verified and pass. We can see report as following
INFO: find district uuid: 56cfcbfe82611dc2ef000001
INFO: determining node uid range: 1000 to 6999
FAIL: anlidom-myapp-1 has a thread: tid:27872, pid:27723 missing from cgroups controller: cpu
INFO: checking presence of tc qdisc
Comment 8 errata-xmlrpc 2016-03-22 12:54:22 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-0489.html

Note You need to log in before you can comment on or make changes to this bug.