Bug 146927 - Exec-Shield feature cause application to crash randomly
Exec-Shield feature cause application to crash randomly
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel (Show other bugs)
i686 Linux
medium Severity high
: ---
: ---
Assigned To: Ingo Molnar
Brian Brock
Depends On:
  Show dependency treegraph
Reported: 2005-02-02 15:19 EST by Paradigm Geophysical
Modified: 2007-11-30 17:07 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2007-10-19 15:07:50 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Contains strace output with and without exec-shield option enabled (117.46 KB, application/octet-stream)
2005-02-04 11:17 EST, Gregory G
no flags Details
Debugging trace for the crashed process including register information and process memory maps (11.04 KB, application/octet-stream)
2005-02-07 03:07 EST, Gregory G
no flags Details
Strace, debugger and maps with MALLOC_CHECK_=3 (19.54 KB, application/octet-stream)
2005-02-07 11:52 EST, Gregory G
no flags Details

  None (edit)
Description Paradigm Geophysical 2005-02-02 15:19:45 EST
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

Description of problem:
The exec-shield functionality appears to cause our applications to 
crash randomly. When we disable the feature our programs work OK. 
The applications are technical applications for the oil
and gas industry. They make extensive use of shared objects.

Version-Release number of selected component (if applicable):
RHEE3U3 x86 : Linux hpnode4 2.4.21-20.ELsmp #1 SMP Wed Aug 18 
20:46:40 EDT 2004 i686 i686 i386 GNU/Linux

How reproducible:

Steps to Reproduce:
1. When our applications in user space with Exec-shield turned on
they experience random crashes. This goes away with the feature 
We would need to supply our apps to demonstrate.

Actual Results:  segmentation voilation

Expected Results:  normal completion

Additional info:

Comment 1 Michael Waite 2005-02-02 15:21:51 EST
Verified as important to our customers and ISVs.
Comment 2 Arjan van de Ven 2005-02-02 15:23:58 EST
what is the stack rlimit set to?
Comment 3 Arjan van de Ven 2005-02-02 15:26:27 EST
also do you JIT code yourself? 
Comment 4 Michael Waite 2005-02-02 15:35:00 EST
Alistair is the Emgineering Manager. The developers are in Isreal and
I will be adding them to this thread once I get their contact information.
Comment 5 Rik van Riel 2005-02-02 17:33:51 EST
AFAIK, Exec-Shield is only turned on if the executable is marked with
a special ELF flag (IIRC PT_GNU_STACK).  For executables without that
flag set, the execshield feature should be disabled.

However, I do not know if the VM layout is also automatically
disabled. The answer to Arjan's stack rlimit question may help us
narrow things down further.
Comment 6 Gregory G 2005-02-04 04:46:12 EST
gregoryg@p4-3 ~--> limit 
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       10240 kbytes
coredumpsize    unlimited
memoryuse       unlimited
vmemoryuse      unlimited
descriptors     1024 
memorylocked    4 kbytes
maxproc         7168 

We do have a code in our s/w that resets the stack limit to 8192KB, however, we 
have tried to remove it and play with the stack limits a little bit.

I would like to send you strace output with and without exec-shield enabled. 
How can I do it?
Comment 7 Michael Waite 2005-02-04 10:30:10 EST
Just attach it to your next bugzilla entry.
"Create a new attachment" is the function you use to create and send an attachment.
Comment 8 Gregory G 2005-02-04 11:17:09 EST
Created attachment 110657 [details]
Contains strace output with and without exec-shield option enabled

What information do you need us to provide you with?
Comment 9 Arjan van de Ven 2005-02-04 11:19:18 EST
one of the key things we'd like to see is the /proc/<pid>/maps file at the time
of the crash. Normally that goes away immediately at crash time, however if you
run the app under gdb at crash time the file will stick around until gdb exists.
Also together with *the same* maps file, we'd like to know the crash address (eg
the EIP of the crash in the app) and in addition the stack pointer (ESP)
Comment 10 Gregory G 2005-02-07 03:07:23 EST
Created attachment 110711 [details]
Debugging trace for the crashed process including register information and process memory maps

As per your request, the 19860.tgz attachement contains the process memory maps
together with a debugging session including backtrace for all threads on the
moment of crash and all registers.

We have means for remote debugging? Is it an option for you?

Comment 11 Gregory G 2005-02-07 11:52:55 EST
Created attachment 110728 [details]
Strace, debugger and maps with MALLOC_CHECK_=3

We have discovered that the program crashes at much earlier stage when using
GLIBC malloc debugging hooks (MALLOC_CHECK_=3). 
Please, find strace output together with a debugging session and process memory
maps on the time of crash with malloc debugging enabled.

Comment 12 RHEL Product and Program Management 2007-10-19 15:07:50 EDT
This bug is filed against RHEL 3, which is in maintenance phase.
During the maintenance phase, only security errata and select mission
critical bug fixes will be released for enterprise products. Since
this bug does not meet that criteria, it is now being closed.
For more information of the RHEL errata support policy, please visit:
If you feel this bug is indeed mission critical, please contact your
support representative. You may be asked to provide detailed
information on how this bug is affecting you.

Note You need to log in before you can comment on or make changes to this bug.