Bug 44619 - g++ -O2 bug in cast expression (80x86)
g++ -O2 bug in cast expression (80x86)
Product: Red Hat Linux
Classification: Retired
Component: gcc (Show other bugs)
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Jakub Jelinek
David Lawrence
Depends On:
  Show dependency treegraph
Reported: 2001-06-14 14:52 EDT by Craig Smith
Modified: 2007-04-18 12:33 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2001-06-19 11:24:58 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:

Attachments (Terms of Use)

  None (edit)
Description Craig Smith 2001-06-14 14:52:25 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.19-7.0.1 i686; en-US; rv:0.9+)

Description of problem:
g++ -O2 for 80x86 produces incorrect assembly code for the
'Cflatten' function in the following snippet (narrowed down and
converted to valid C from a large C++ program):

#include <stdio.h>

typedef short int16;
typedef long int32;

typedef struct {
	int16 v, h;
	} Point;

int32 Cflatten(Point *pp) {
	Point p;
	p.v = pp->v;
	p.h = pp->h;
	return (*((int32*)));

int main() {
	Point p = { 0x1234, 0x5678 };
	int32 f;

	f = Cflatten();
	printf("%08x\n", f); /* should print 56781234 on a little-endian machine */

The Cflatten function generates the following x86 assembly:

        pushl   %ebp
        movl    %esp, %ebp
        pushl   %eax
        movl    8(%ebp), %eax
        movw    (%eax), %ax
        movw    %ax, -4(%ebp)
        movl    -4(%ebp), %eax

which leaves the upper half of %eax uninitialized.

How reproducible:

Steps to Reproduce:
1.Compile the above code fragment with g++ -O2
2.Run and observe the output is bfff1234 instead of 56781234
3.Re-compile without -O2 and observe the correct output.  

Actual Results:  Output is "bfff1234"

Expected Results:  Output should be "56781234" (on a little-endian machine)

Additional info:

The bug does not occur if you turn off the strict-aliasing optimization,
e.g. "g++ -O2 -fno-strict-aliasing".  It also does not occur if you
use gcc instead of g++ to compile.  If you change the initialization of
Point p to use struct initialization syntax, or make p volatile, the
problem goes away.

Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/2.96/specs
gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-81)

I haven't tried to see if this bug exists in a gcc 3.0 pre-release.
Comment 1 Michael Schwendt 2001-06-15 14:37:02 EDT
See bug #36862.
Comment 2 Craig Smith 2001-06-15 17:51:25 EDT
Bug 36862 referenced above is clearly related, yet this bug is closed as "not a
bug".  But there clearly IS a problem here.  In my case, the entire 32-bit word
should be read and returned, but it's returning half-uninitialized memory.
Comment 3 Michael Schwendt 2001-06-16 06:40:43 EDT
Apart from these two typos in your example,

-return (*((int32*)));
+return (*((int32*)&p));

-f = Cflatten();
+f = Cflatten(&p);

and gcc -O2 returning 0x0000 in the upper 16 bits, the *((int32*)&p) cast is bad
code and should be fixed. For an introduction to the problem, see "info gcc",
node "Optimize Options", on -fstrict-aliasing, or search the web on type-punning.
Comment 4 Craig Smith 2001-06-18 13:27:32 EDT
Apologies for the 'typos'.  I actually cut-and-pasted that into the web form
from working code, but apparently the ampersands got eaten and I didn't notice.

The results of web-searches on -fstrict-aliasing and type-punning consisted
mostly of people complaining that code didn't work with gcc -fstrict-aliasing
and being told by people from the gcc team that the code was bad and needed to
be fixed. There was very little information on why it is considered valid for
the compiler to make this optimization.

There were references to the ANSI C Standard ISO-9899:1990 Sections 3.3 and, but the actual standard is impossible to find.  Books containing it are
out of print, and ansi.org appears to only have the 1999 standard available.
If someone could e-mail me or append here just the relevant sections of the
standard, or point me where I could find them, I'd appreciate it.

I realize that code of the sort *(int32*)&p is inherently non-portable, but on a
known architecture where the size of the pointers are the known to be the same,
I find it shocking that compiler-writers would allow code like this to break
without so much as a warning message. 

While not "clean", it is sometimes necessary.  For example, some of our code
which does this kind of thing is reading an arbitrary block of data over a TCP
connection, and interpreting the content by dynamically parsing a
type-descriptor at run-time.  If deferencing pointer casts does not work, how
else is one supposed to do this?  And even if declared technically legal by the
standard, it seems like turning on an optimization by default in -O2 with the
potential to break lots of code is a bad idea.

And I'm sure I'm not alone.  Here's a nice quote on the subject from Linux
Torvalds (Linux Kernel mailing list, Dec 2000):
> I hope that's another thing that the gcc people fix by the time they do a
> _real_ release. Anobody who thinks that "-fstrict-aliasing" being on by
> default is a good idea is probably a compiler person who hasn't seen real 
> code. 
Comment 5 Michael Schwendt 2001-06-19 11:24:55 EDT
> I realize that code of the sort *(int32*)&p is inherently
> non-portable, but on a known architecture where the size
> of the pointers are the known to be the same,

Not the size of the pointers is the problem, but finding out when to update the
object pointed to in memory, i.e. whether the value may be cached until it is
accessed the next time.

Checking out the C9X Rational or the C9X Committee Draft might be an idea. IIRC,
it is somewhere in section 6.5. I've copied this from my Sent folder. I think
it's an excerpt from the C9X Rational:

In principle, then, aliasing only need be allowed for when the lvalues all have
the same type.  In practice, the C89 Committee recognized certain prevalent

*    The lvalue types may differ in signedness.  In the common range, a signed
integer type and its unsigned variant have the same representation; and it was
felt that an appreciable body of existing code is not "strictly typed" in this area.

*    Character pointer types are often used in the bytewise manipulation of
objects; a byte stored through such a character pointer may well end up in an
object of any type.

*    A qualified version of the object's type, though formally a different type, 
provides the same interpretation of the value of the object.
Comment 6 Jakub Jelinek 2001-06-25 06:04:12 EDT
Thanks Michael for explaining instead of myself.
You basically have 3 options: either decide you don't want to fix your code,
but then you should stick -fno-strict-aliasing into your Makefiles. Or make sure
one of the types is a char (whatever signedness) pointer, ie. you can cast
a pointer to anything to a char pointer, or a char pointer to pointer to whatever
and access through that (note that the problem is not the casting itself, but
accessing the same memory location with two incompatible types).
The third option is to use unions, e.g. above you could either
int32 Cflatten(Point *pp) {
        Point p;
        p.v = pp->v;
        p.h = pp->h;
        return ((union { Point p; int32 i; } *)&p)->i;
int32 Cflatten(Point *pp) {
        union { Point p; int 32 i; } p;
        p.p.v = pp->v;
        p.p.h = pp->h;
        return p.i;
Comment 7 Craig Smith 2001-06-25 14:30:23 EDT
Well, I'm partially mollified by the exception for 'char', which would make most
code which does this sort of thing continue to work correctly.

I still agree with Linus that it was a bad idea to turn on this optimization bu
default in -O2.  The problem is that there is no way to tell when the code does
not compile as expected.

I would prefer to fix our source code over using -fno-strict-aliasing, but I
don't know how to go about finding all such code in a deterministic way.  Would
you consider adding an option to gcc to generate a warning when code is
encountered that due to aliasing might have a different effect than the user
would expect?

Note You need to log in before you can comment on or make changes to this bug.