Bug 57760 - gcc-generated code accesses stack below sp
Summary: gcc-generated code accesses stack below sp
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: gcc
Version: 7.2
Hardware: i686
OS: Linux
medium
high
Target Milestone: ---
Assignee: Jakub Jelinek
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2001-12-21 09:17 UTC by Xavier Leroy
Modified: 2007-04-18 16:38 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2002-07-01 16:51:26 UTC
Embargoed:


Attachments (Terms of Use)
excerpt from Colin Plumb's MD5 implementation demonstrating wrong generated code (4.12 KB, text/plain)
2001-12-21 09:18 UTC, Xavier Leroy
no flags Details
excerpt from NSS 3.3.2's SHA1 implementation demonstrating wrong generated code (8.92 KB, text/plain)
2001-12-22 00:26 UTC, Wan-Teh Chang
no flags Details
diffs between the assembler files generated by 'gcc -S -O2 -fPIC sha1.c' on 7.1 and 7.2 (595 bytes, patch)
2001-12-22 00:34 UTC, Wan-Teh Chang
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2002:055 0 contract SHIPPED_LIVE Updated version of GCC 2.96-RH now available 2002-04-02 05:00:00 UTC

Description Xavier Leroy 2001-12-21 09:17:07 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2)
Gecko/20010726 Netscape6/6.1

Description of problem:
In certain leaf functions, the code generated by gcc does not allocate a
new stack frame, i.e. it does not subtract from %esp in the function
prelude, yet the remainder of the code accesses stack-allocated local
variables (at negative offsets w.r.t %ebp).  This results in accesses to
stack locations below the stack pointer %esp.  These locations can be wiped
at any time by a signal handler, causing the program to crash.  Also, this
can invalidate the stack growing heuristic in the kernel, causing the
program to be killed on a segmentation violation.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Compile the attached md5.c file with "gcc -O -S"
2. Look at md5.s, function MD5Transform, and notice the lack of
a "subl $NNN, %esp" instruction in the prelude, and the accesses
" movl %eax, -16(%ebp)" later in the code.


Additional info:

This is the same bug as #55568, but with a much smaller code fragment to
reproduce it.

Comment 1 Xavier Leroy 2001-12-21 09:18:22 UTC
Created attachment 41168 [details]
excerpt from Colin Plumb's MD5 implementation demonstrating wrong generated code

Comment 2 Wan-Teh Chang 2001-12-22 00:15:34 UTC
We also ran into this bug when compiling the SHA1 code in the
Network Security Services (NSS) libraries on Red Hat Linux 7.2.
See http://bugzilla.mozilla.org/show_bug.cgi?id=116327

The code generated by gcc 2.96-81 on Red Hat Linux 7.1 is good.

I will attach an excerpt of the SHA1 code in NSS.


Comment 3 Wan-Teh Chang 2001-12-22 00:26:59 UTC
Created attachment 41207 [details]
excerpt from NSS 3.3.2's SHA1 implementation demonstrating wrong generated code

Comment 4 Wan-Teh Chang 2001-12-22 00:34:10 UTC
Created attachment 41208 [details]
diffs between the assembler files generated by  'gcc -S -O2 -fPIC sha1.c' on 7.1 and 7.2

Comment 5 Jakub Jelinek 2002-02-14 23:13:02 UTC
Should be fixed in gcc-2.96-103, currently at:
ftp://people.redhat.com/jakub/gcc/2.96-103/

Comment 6 Wan-Teh Chang 2002-02-15 00:33:43 UTC
Jakub,

This is good news.  Thanks.

We will need to wait for an official Red Hat Linux 7.2 update
that includes this fix though.  Do you know when that will be
available?

Comment 7 Wan-Teh Chang 2002-03-06 02:38:59 UTC
I verified that gcc 2.96-103 generates the same code from my sha1.c
test case as gcc 2.96-81.

Comment 8 Christopher Blizzard 2002-06-29 21:42:01 UTC
So this is not fixed?

Comment 9 Wan-Teh Chang 2002-07-01 16:51:22 UTC
This is fixed.  gcc 2.96-81 and gcc 2.96-103 generate the
correct code.  It's 2.96-98 (the gcc in Red Hat Linux 7.2)
that has the bug.

Comment 10 Christopher Blizzard 2002-07-01 18:53:18 UTC
OK, then the bug is fixed.


Note You need to log in before you can comment on or make changes to this bug.