Bug 144172 - binfmt_aout DoS
Summary: binfmt_aout DoS
Alias: None
Product: Red Hat Enterprise Linux 2.1
Classification: Red Hat
Component: kernel   
(Show other bugs)
Version: 2.1
Hardware: ia64
OS: Linux
Target Milestone: ---
Assignee: Don Howard
QA Contact: Brian Brock
Keywords: Security
Depends On:
Blocks: 143573
TreeView+ depends on / blocked
Reported: 2005-01-04 21:46 UTC by Josh Bressers
Modified: 2007-11-30 22:06 UTC (History)
3 users (show)

Fixed In Version: RHSA-2006-0190
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2006-02-01 17:44:31 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2006:0190 normal SHIPPED_LIVE Important: kernel security update 2006-02-01 05:00:00 UTC

Description Josh Bressers 2005-01-04 21:46:18 UTC
Marcelo Tosatti brought this to the attention of vendor-sec

The recent binfmt_aout v2.6 backport changes also fix a DoS:

ChangeSet@1.1527.1.13, 2004-12-16 16:06:31-02:00, chrisw@osdl.org
  [PATCH] a.out: error check on set_brk

  It's possible for do_brk() to fail during set_brk() when exec'ing and
  a.out.  This was noted with Florian's a.out binary and overcommit set to

  Capture this error and terminate properly.

ChangeSet@1.1527.1.16, 2004-12-17 21:45:58-02:00, chrisw@osdl.org
  [PATCH] Backport of 2.6 fix to insert_vm_struct to make it return an error
rather than BUG().

  Backport of 2.6 fix to insert_vm_struct to make it return an error
  rather than BUG().  This eliminates a user triggerable BUG() when user
  created a large vma that overlapped with arg pages during exec (could be
  triggered with a.out on i386 and x86_64 and elf on ia64).

  Signed-off-by: Chris Wright <chrisw@osdl.org>

Comment 1 Ernie Petrides 2005-01-04 22:12:35 UTC
Jim, note my comment in bug 144153.  I'm not sure if it applies to RHEL2.1.

Comment 2 Jim Paradis 2005-01-17 21:26:42 UTC
The second patchset above has already been applied to AS2.1-U7 (see bug 144785).
 The first one has not, and is applicable to AS2.1-ia64.

Comment 3 Don Howard 2005-07-27 22:59:42 UTC
It looks like the first pachset (the do_brk patch) is not applicable to
pensacola nor to derry.

On pensacola (with the binfmt_aout.o kernel module loaded) the test program is
killed with SIGSEGV.

On derry, it appears that we don't ship a binfmt_aout module (correct me if that
is wrong).

The test binary I used is generated as follows:

perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout

Comment 4 Don Howard 2005-10-17 18:46:22 UTC
On code inspection, it looks like the set_brk patch is needed in derry for
binfmt_elf and binfmt_aout, though we don't appear to support aout on derry.

Comment 8 Red Hat Bugzilla 2006-02-01 17:44:31 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.