The issue below only affected 2.6 kernels, however the vulnerable code is part of Enterprise Linux 3 due to the backport of nptl. Therefore this issue may also affect RHEL3. +++ This bug was initially created as a clone of Bug #174078 +++ The auto-reap of child processes in Linux kernel 2.6 before 2.6.15 includes processes with ptrace attached, which leads to a dangling ptrace reference and allows local users to cause a denial of service (crash). Fixed upstream by http://linux.bkbits.net:8080/linux-2.6/cset@437a0568g4lPMynwmUw1ajvC2ZroDg
Created attachment 122345 [details] Proposed patch
Mark, would it be possible to dig up a reproducer for this? There is an extra check (tsk->state != TASK_STOPPED) in RHEL3 that is not in 2.6 that might preclude the vulnerability. The patch in comment #1 carries the risk of introducing a DoS regression (kernel memory exhaustion due to not cleaning up task_struct's), so we need to be 100% confident that we're fixing a real problem and that our fix is correct. Thanks in advance.
Thanks for the info, Mark. The reproducer gets an error on the clone() syscall on RHEL3, because the flags CLONE_THREAD and CLONE_DETACHED must either both be set or both be clear. So, I altered the test program to try both cases (both flags set and both flags clear), and in neither case was there a problem on my test system. Further, the info in the thread in comment #4 indicates that the upstream failure mode is that a BUG_ON() is hit in release_task(), and this does not exist in RHEL3. Rather, there is a test for the condition that prevents the associated block of code from being executed in the problematic case. The bottom line is that RHEL3 is not vulnerable to this bug.