Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

For bugs related to Red Hat Enterprise Linux 2.1 product line. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 144172

Summary:	binfmt_aout DoS
Product:	Red Hat Enterprise Linux 2.1	Reporter:	Josh Bressers <bressers>
Component:	kernel	Assignee:	Don Howard <dhoward>
Status:	CLOSED ERRATA	QA Contact:	Brian Brock <bbrock>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	2.1	CC:	jparadis, peterm, riel
Target Milestone:	---	Keywords:	Security
Target Release:	---
Hardware:	ia64
OS:	Linux
Whiteboard:
Fixed In Version:	RHSA-2006-0190	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2006-02-01 17:44:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	143573

Description Josh Bressers 2005-01-04 21:46:18 UTC

Marcelo Tosatti brought this to the attention of vendor-sec

The recent binfmt_aout v2.6 backport changes also fix a DoS:

ChangeSet.1.13, 2004-12-16 16:06:31-02:00, chrisw
  [PATCH] a.out: error check on set_brk

  It's possible for do_brk() to fail during set_brk() when exec'ing and
  a.out.  This was noted with Florian's a.out binary and overcommit set to
  0.

  Capture this error and terminate properly.

ChangeSet.1.16, 2004-12-17 21:45:58-02:00, chrisw
  [PATCH] Backport of 2.6 fix to insert_vm_struct to make it return an error
rather than BUG().

  Backport of 2.6 fix to insert_vm_struct to make it return an error
  rather than BUG().  This eliminates a user triggerable BUG() when user
  created a large vma that overlapped with arg pages during exec (could be
  triggered with a.out on i386 and x86_64 and elf on ia64).

  Signed-off-by: Chris Wright <chrisw>

Comment 1 Ernie Petrides 2005-01-04 22:12:35 UTC

Jim, note my comment in bug 144153.  I'm not sure if it applies to RHEL2.1.

Comment 2 Jim Paradis 2005-01-17 21:26:42 UTC

The second patchset above has already been applied to AS2.1-U7 (see bug 144785).
 The first one has not, and is applicable to AS2.1-ia64.

Comment 3 Don Howard 2005-07-27 22:59:42 UTC

It looks like the first pachset (the do_brk patch) is not applicable to
pensacola nor to derry.

On pensacola (with the binfmt_aout.o kernel module loaded) the test program is
killed with SIGSEGV.

On derry, it appears that we don't ship a binfmt_aout module (correct me if that
is wrong).

The test binary I used is generated as follows:

perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout

Comment 4 Don Howard 2005-10-17 18:46:22 UTC

On code inspection, it looks like the set_brk patch is needed in derry for
binfmt_elf and binfmt_aout, though we don't appear to support aout on derry.

Comment 8 Red Hat Bugzilla 2006-02-01 17:44:31 UTC

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2006-0190.html