CVE-2005-3271 states "Exec in Linux kernel 2.6 does not properly clear posix-timers in multi-threaded environments, which results in a resource leak and could allow a large number of multiple local users to cause a denial of service by using more posix-timers than specified by the quota for a single user." This code was included in RHEL3 due to a backport, in linux-2.4.20-o1-nptl.patch and therefore may be vulnerable to this issue (needs investigation from kernel team). This was originally fixed with this commit: http://linux.bkbits.net:8080/linux-2.6/cset@414b332fsZQvEUsfzKJIo-q2_ZH0hg however the code got refactored shortly afterwards: http://linux.bkbits.net:8080/linux-2.6/cset@4174ac1exFxpMg163OsRuPZLQrlBKg We currently do not have any more information on this issue, the CVE looks like it has been created completely from the original commit message.
Looks to me like RHEL3 does have this vulnerability, although I'd prefer to see it fixed without doing any signal struct rework as is shown by the 2nd link above. The upstream 2.6 signal data struct handling is somewhat different from RHEL3, which is why mucking around with that part would be too risky. It might be that the change shown by the 1st link is sufficient to address the vulnerability. PeterS, I'm tentatively assigning this to you, but I might have some time next week to take this one on myself.
I've determined that the leaked data structures in 2.6 are used in the implementation of POSIX.1b interval timers, which are instantiated with the timer_create() system call. RHEL3 does not support these timers nor their associated system calls, and thus it is not vulnerable to this problem. Closing as NOTABUG.