Bug 57760 - gcc-generated code accesses stack below sp
gcc-generated code accesses stack below sp
Status: CLOSED ERRATA
Product: Red Hat Linux
Classification: Retired
Component: gcc (Show other bugs)
7.2
i686 Linux
medium Severity high
: ---
: ---
Assigned To: Jakub Jelinek
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2001-12-21 04:17 EST by Xavier Leroy
Modified: 2007-04-18 12:38 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2002-07-01 12:51:26 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
excerpt from Colin Plumb's MD5 implementation demonstrating wrong generated code (4.12 KB, text/plain)
2001-12-21 04:18 EST, Xavier Leroy
no flags Details
excerpt from NSS 3.3.2's SHA1 implementation demonstrating wrong generated code (8.92 KB, text/plain)
2001-12-21 19:26 EST, Wan-Teh Chang
no flags Details
diffs between the assembler files generated by 'gcc -S -O2 -fPIC sha1.c' on 7.1 and 7.2 (595 bytes, patch)
2001-12-21 19:34 EST, Wan-Teh Chang
no flags Details | Diff

  None (edit)
Description Xavier Leroy 2001-12-21 04:17:07 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2)
Gecko/20010726 Netscape6/6.1

Description of problem:
In certain leaf functions, the code generated by gcc does not allocate a
new stack frame, i.e. it does not subtract from %esp in the function
prelude, yet the remainder of the code accesses stack-allocated local
variables (at negative offsets w.r.t %ebp).  This results in accesses to
stack locations below the stack pointer %esp.  These locations can be wiped
at any time by a signal handler, causing the program to crash.  Also, this
can invalidate the stack growing heuristic in the kernel, causing the
program to be killed on a segmentation violation.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Compile the attached md5.c file with "gcc -O -S"
2. Look at md5.s, function MD5Transform, and notice the lack of
a "subl $NNN, %esp" instruction in the prelude, and the accesses
" movl %eax, -16(%ebp)" later in the code.


Additional info:

This is the same bug as #55568, but with a much smaller code fragment to
reproduce it.
Comment 1 Xavier Leroy 2001-12-21 04:18:22 EST
Created attachment 41168 [details]
excerpt from Colin Plumb's MD5 implementation demonstrating wrong generated code
Comment 2 Wan-Teh Chang 2001-12-21 19:15:34 EST
We also ran into this bug when compiling the SHA1 code in the
Network Security Services (NSS) libraries on Red Hat Linux 7.2.
See http://bugzilla.mozilla.org/show_bug.cgi?id=116327

The code generated by gcc 2.96-81 on Red Hat Linux 7.1 is good.

I will attach an excerpt of the SHA1 code in NSS.
Comment 3 Wan-Teh Chang 2001-12-21 19:26:59 EST
Created attachment 41207 [details]
excerpt from NSS 3.3.2's SHA1 implementation demonstrating wrong generated code
Comment 4 Wan-Teh Chang 2001-12-21 19:34:10 EST
Created attachment 41208 [details]
diffs between the assembler files generated by  'gcc -S -O2 -fPIC sha1.c' on 7.1 and 7.2
Comment 5 Jakub Jelinek 2002-02-14 18:13:02 EST
Should be fixed in gcc-2.96-103, currently at:
ftp://people.redhat.com/jakub/gcc/2.96-103/
Comment 6 Wan-Teh Chang 2002-02-14 19:33:43 EST
Jakub,

This is good news.  Thanks.

We will need to wait for an official Red Hat Linux 7.2 update
that includes this fix though.  Do you know when that will be
available?
Comment 7 Wan-Teh Chang 2002-03-05 21:38:59 EST
I verified that gcc 2.96-103 generates the same code from my sha1.c
test case as gcc 2.96-81.
Comment 8 Christopher Blizzard 2002-06-29 17:42:01 EDT
So this is not fixed?
Comment 9 Wan-Teh Chang 2002-07-01 12:51:22 EDT
This is fixed.  gcc 2.96-81 and gcc 2.96-103 generate the
correct code.  It's 2.96-98 (the gcc in Red Hat Linux 7.2)
that has the bug.
Comment 10 Christopher Blizzard 2002-07-01 14:53:18 EDT
OK, then the bug is fixed.

Note You need to log in before you can comment on or make changes to this bug.