Bug 112592 - prelinking causes mmap() to SEGV in some cases
prelinking causes mmap() to SEGV in some cases
Product: Fedora
Classification: Fedora
Component: prelink (Show other bugs)
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Jakub Jelinek
Depends On:
  Show dependency treegraph
Reported: 2003-12-23 16:36 EST by Kars de Jong
Modified: 2007-11-30 17:10 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2003-12-23 19:14:18 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Example test program (297 bytes, text/plain)
2003-12-23 16:37 EST, Kars de Jong
no flags Details

  None (edit)
Description Kars de Jong 2003-12-23 16:36:51 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4.1)

Description of problem:
I have upgraded 3 systems previously running Red Hat 9 to Fedora Core
1. I also installed prelink on all of these systems. Today I noticed
that one of my own applications no longer works: it gets a
segmentation violation. I tried running it on the other systems, and
to my surprise it worked fine on one system while it also crashed on

The crash appears to happen inside the C library part of mmap(). When
I use strace I see mmap2() being called and returning successfully,
but when using ltrace the mmap() function hasn't returned yet.

When linking my program statically it worked fine however...
Also, when I disabled prelinking (after I ran /etc/cron.daily/prelink
by hand) it worked fine too. Please note that the program itself
wasn't prelinked yet.

I made a small program to demonstrate the effect.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Make sure prelinking is enabled
2. Compile mmap_bug.c with "gcc -Wall -O -g -o mmap_bug mmap_bug.c"
2. Run it with "./mmap_bug"
4. Repeat with ltrace or strace or gdb    

Actual Results:  The program crashes on some systems.

Expected Results:  It should have worked just fine (no output expected).

Additional info:
Comment 1 Kars de Jong 2003-12-23 16:37:58 EST
Created attachment 96684 [details]
Example test program
Comment 2 Kars de Jong 2003-12-23 16:54:59 EST
Hmm, it looks like I spoke too soon...
It still crashes sometimes, even with prelinking disabled.

After some more investigation, it looks like the Exec-shield
functionality has something to do with it too. On my system with
Exec-shield enabled and prelink disabled, it still crashes. I
understand why too now, because the fixed memory mapping of 4 MB at
address 0 my program makes sometimes overlaps with the place where the
C library is mapped. Using setarch fixes this issue, and also
disabling Exec-shield globally. However, this doesn't work for a
prelinked library.

I guess I just can't use prelinking in this case, unless I can somehow
make it exclude certain memory ranges... (I *need* the memory mapping
at address 0 since this is some kind of emulator).
Comment 3 Jakub Jelinek 2003-12-23 19:14:18 EST
Exec-shield places libraries below the binary (starting from
1MB+4KB up to the beginning of the binary).
If your program needs to use MAP_FIXED on a region not given by the kernel to you
(e.g. MAP_FIXED on previously returned address from mmap without MAP_FIXED
if the region is smaller or equal is always ok), then you need to tell the system
about it.
The best way is to create a p_flags 0 PT_LOAD segment with p_filesz 0
and p_memsz equal to the size of the region you want to reserve,
p_vaddr equal to the start address.
Then it will work even with exec-shield, prelinking etc.
ATM it is not exactly trivial to create such PT_LOAD segments, it needs
quite many changes to the linker script (see PHDRS linker script command),
but it is on my TODO list to simplify this.

Alternatively, you need to disable exec-shield for that particular
program.  With rawhide glibc and maybe already rawhide kernel (am not sure
whether the patch is in there already) if you build your program as PIE
and disable exec-shield for it, it should load even prelinked libraries

Or, if you don't want exec-shield system-wide, you can add --no-exec-shield
option to /etc/sysconfig/prelink and force reprelinking.

Note You need to log in before you can comment on or make changes to this bug.