Description of problem:
IBM reported this to me. Using their attached test program, it can be
seen that if a call to shmat is made, and an address is passed to it
that is already in use in the process address space, the shmat call
will silently overwrite the previous allocation with the shared memory.
Version-Release number of selected component (if applicable):
all AS2.1 kernels
Steps to Reproduce:
1. Compile the attached program
2. Make sure to allow sufficient space for shared memory allocations
by bumping up the value of /proc/sys/kernel/shmmax (the test program
will inform as to how much based on the allocation size).
3. Run the compiled program
Both malloc and shmat succede. Looking at the maps file of the
process, you can see the address space of the malloc is overwritten by
the mapping of the shared memory when the call to shmat is made.
shmat call should fail, and return EINVAL.
Technically there appears to be nothing in the SUS that indicates this
should fail, but it certainly seems broken to sliently overwrite
memory mappings like this. 2.6 and the RHEL3 2.4 tree have changed
this behavior to return EINVAL if a already in use memory address is
passed to shmat and the SHM_REMAP flag is not set.
Created attachment 105588 [details]
test program from IBM to recreate this issue
this is the test program for use in the recreation
Created attachment 105589 [details]
patch to make shmat return -EINVAL if used addr is passed to shmat
This is a 2.6 backport of the patch to shmat that causes the system call to
return EINVAL in the event that an in use address is passed in and the
SHM_REMAP flag is not set.
The reported behavior is technically compliant with the POSIX standard, even
though it may be inconsistent with the way other kernels work. More
importantly, changing the behavior at this time entails unacceptable risk;
returning error codes where we did not do so before can cause a lot of existing
software to break. Therefore, we are leaving this code as-is.