*** This bug has been split off bug 144361 ***
------- Original comment by Josh Bressers (Security Response Team) on 2005.01.06
From the lkml mailing list.
It is possible that an improperly formed a.out binary can cause a
kernel-oops, which if executed in a loop will ead fd's and memory.
It seems that you have to turn on memory overcommit on for this to
davej says we didn't enable a.out binaries on FC3 and RHEL4. This
will be undone in the future though.
We do not turn on memory overcommit by default in RHEL2.1 or 3.
This should make the impact of this issue significantly mitigated
since it seems the issue is only exploitable when overcommit
is on (sysctl -w vm.overcommit_memory=1). However note that
we do tell customers how to turn this on.
fixed=2.6.10 (20041116 cset@419aaba8xdR0decwoMnVpt3G8_f8kQ)
not fixed for 2.4 (as at Nov24)
This is NOTABUG for AS2.1-i386 since it is based on 2.4.9 which handles vma's
differently. The poisoned binary exits with an error rather than crashing, so
there's no DoS.