Bug 2809 - egcs function return convention incompatible with gcc
Summary: egcs function return convention incompatible with gcc
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: egcs
Version: 6.0
Hardware: i386
OS: Linux
Target Milestone: ---
Assignee: David Lawrence
QA Contact:
Depends On:
TreeView+ depends on / blocked
Reported: 1999-05-14 04:27 UTC by ctm
Modified: 2008-05-01 15:37 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 1999-05-30 21:20:34 UTC

Attachments (Terms of Use)

Description ctm 1999-05-14 04:27:43 UTC
I'm flagging this as security priority because there's a
chance that this is exploitable.

As it's configured in RH5.2 and RH6.0, egcs uses a different
way to return structs that are larger than 32 bytes than the
version of gcc shipped on RH5.2.  Since the system libraries
were compiled with gcc under RH5.2 and are now compiled
under egcs, this has several reprecussions:

the compat-egcs in RH6.0 will *NOT* create binaries that can
be safely run on RH5.2, if any of the code being compiled
calls functions that return structs larger than 32 bits,
such as the various DBM calls.  Doing so causes stack
corruption that may result in immediate program death, may
result in incorrect values or may result in code that
sometimes works and sometimes fails, depending on whether or
not any signals are processed during critical windows.

This also means that various software that was compiled on
Red Hat 5.x systems may fail to work (or may become flakey)
on RH6.0, even though the code didn't do anything out of the
ordinary (other than calling functions like the ndbm

I first ran into this problem when I tried to use a copy of
Executor (our product) compiled on a RH5.x system on a RH6.0
pre-release.  I didn't initially realize what the problem
was and mis-attributed it to the glibc2.0 -> glibc2.1
upgrade, but it's easy to demonstrate this problem on a
stock RH5.2 machine, just by using the copy of gcc and egcs
that's on RH5.2.

I've written up a very small test case that I can e-mail to
whomever is interested in this bug.

Although I'm flagging this as a Red Hat 6.0 bug, Red Hat is
not alone.  SuSE 6.0 also made this switch and is similarly
bitten.  I haven't had a chance to look at the egcs source
to find out whether the change in function return
methodology is something that can trivially be changed at
configuration time or not, but if this change was
deliberate, it should be accompanied by MAJOR WARNINGS OF
INCOMPATIBILITY.  But since most programmers don't write
functions that return structs that are larger than 32-bits,
my guess is the change was not deliberate and was simply

Comment 1 Jeff Johnson 1999-05-30 21:20:59 UTC
If EGCS development fixes this problem, then Red Hat will.
Otherwise, use EGCS for all compilation.

Note You need to log in before you can comment on or make changes to this bug.