Marcelo Tosatti brought this to the attention of vendor-sec
The recent binfmt_aout v2.6 backport changes also fix a DoS:
ChangeSet@1.1527.1.13, 2004-12-16 16:06:31-02:00, email@example.com
[PATCH] a.out: error check on set_brk
It's possible for do_brk() to fail during set_brk() when exec'ing and
a.out. This was noted with Florian's a.out binary and overcommit set to
Capture this error and terminate properly.
ChangeSet@1.1527.1.16, 2004-12-17 21:45:58-02:00, firstname.lastname@example.org
[PATCH] Backport of 2.6 fix to insert_vm_struct to make it return an error
rather than BUG().
Backport of 2.6 fix to insert_vm_struct to make it return an error
rather than BUG(). This eliminates a user triggerable BUG() when user
created a large vma that overlapped with arg pages during exec (could be
triggered with a.out on i386 and x86_64 and elf on ia64).
Signed-off-by: Chris Wright <email@example.com>
Jim, note my comment in bug 144153. I'm not sure if it applies to RHEL2.1.
The first changeset referenced above is applicable to AS2.1-i386. The second
one is not, since it handles insert_vm_struct somewhat differently.
The set_brk() patch is needed for aout and elf on pensacola.
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.