Bug 488956 - pointer arithmetic weirdness on 64-bit systems (does not return multiple of sizeof(object))
pointer arithmetic weirdness on 64-bit systems (does not return multiple of s...
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: gcc (Show other bugs)
5.0
x86_64 Linux
low Severity medium
: rc
: ---
Assigned To: Jakub Jelinek
BaseOS QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-03-06 09:20 EST by Samo Dadela
Modified: 2009-03-09 12:09 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-03-09 11:44:46 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
test proggy (579 bytes, text/x-csrc)
2009-03-06 09:20 EST, Samo Dadela
no flags Details

  None (edit)
Description Samo Dadela 2009-03-06 09:20:45 EST
Created attachment 334295 [details]
test proggy

Description of problem:
Subtracting two object pointers returns the difference in bytes instead of multiples of sizeof(object). That happens only on 64-bit systems. Problem was detected on RHAS 5.0, RHAS 5.3 (and also SUSE 10.0)

Version-Release number of selected component (if applicable):


How reproducible:
gcc -o dk dk.c 

Steps to Reproduce:
1. Run dk on 32-bit system
2. Run dk on 64-bit system. 
3. Compare results
  
Actual results:
Fedora 10 (32-bit)
$ ./dk
stack:0xbfbe49cc   heap:0x990a008
y:0x990a070   x:0x990a008  y-x:0x68
p1:0x990a140   p2:0x990a1d8  p2-p1:0x1        <- note, p2-p1
pg1:0xbfbe493c   pg2:0xbfbe48ac  pg2-pg1:0xffffffff

RHAS 5.0 (64-bit) 
# ./dk
stack:0x7fff9ac444a0   heap:0xac6d010
y:0xac6d080   x:0xac6d010  y-x:0x70
p1:0xac6d150   p2:0xac6d1f0  p2-p1:0x86bca1af286bca1c
pg1:0x7fff9ac44400   pg2:0x7fff9ac44360  pg2-pg1:0x79435e50d79435e4

RHAS 5.3 (64-bit)
# ./dk
stack:0x7fff7d3dfc80   heap:0x1c87010
y:0x1c87080   x:0x1c87010  y-x:0x70
p1:0x1c87150   p2:0x1c871f0  p2-p1:0x86bca1af286bca1c <- note p2-p1
pg1:0x7fff7d3dfbe0   pg2:0x7fff7d3dfb40  pg2-pg1:0x79435e50d79435e4

SUSE 10 (64-bit):
./dk  
stack:0x7fff156d4060   heap:0x501010
y:0x501080   x:0x501010  y-x:0x70
p1:0x501150   p2:0x5011f0  p2-p1:0x86bca1af286bca1c
pg1:0x7fff156d3fc0   pg2:0x7fff156d3f20  pg2-pg1:0x79435e50d79435e4


Expected results:
p2-p1 should contain the difference in multiples of sizeof(struct mystr) like on 32-bit systems (HPUX 11.32 also returns the same). 

Additional info:
This behaviour causes problems when doing pointer arithmetic. We were using a c++ (stl) set that had the 'less' operator defined like this:

d = (obj *)a1 - (obj *)a2; 
return d; 

and there were arithmetic overflows/underflows which caused the set to be incorrectly ordered for certain functions.
Comment 1 Samo Dadela 2009-03-06 09:32:09 EST
Forgot: all gcc-a are vanilla. 

Here are the versions: 
Fedora 10 (32-bit)
 gcc (GCC) 4.3.2 20081105 (Red Hat 4.3.2-7)

RHAS 5.3:
 gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-44)

RHAS 5.0: 
 gcc (GCC) 4.3.2 20081105 (Red Hat 4.3.2-7)
Comment 2 Jakub Jelinek 2009-03-09 11:44:46 EDT
This is not valid C (nor C++).  ISO C99 says in 9.5.6/9:
"When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; the result is the difference of the subscripts of the two array elements.".

The compiler assumes it is the case and optimizes based on this (can use multiplication instead of division).

So, you can't compare using such a less operator pointers that don't point to the same array (or one past the last element of it).  You should cast them to uintptr_t and compare as integers instead or something similar.
Comment 3 Samo Dadela 2009-03-09 12:09:15 EDT
To recap: 
> "When two pointers are subtracted, both shall point to elements of the same
> array object, or one past the last element of the array object; the result is
> the difference of the subscripts of the two array elements.".

So you are saying that in all other cases the result is undefined and that I've should not have used that way of comparing pointers from the start? 

Was it just a coincidence that this worked on 32-bit systems? 
Do you have maybe an explanation on why it worked on 32-bit systems and why it doesn't on 64-bit? 

The code I was porting originated on HPUX... and it also works the same on Linux 32-bit... however on Linux 64 I get this strange behaviour... Hmm...

Note You need to log in before you can comment on or make changes to this bug.