Bug 146927 - Exec-Shield feature cause application to crash randomly
Summary: Exec-Shield feature cause application to crash randomly
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel
Version: 3.0
Hardware: i686
OS: Linux
medium
high
Target Milestone: ---
Assignee: Ingo Molnar
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2005-02-02 20:19 UTC by Paradigm Geophysical
Modified: 2007-11-30 22:07 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-10-19 19:07:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Contains strace output with and without exec-shield option enabled (117.46 KB, application/octet-stream)
2005-02-04 16:17 UTC, Gregory G
no flags Details
Debugging trace for the crashed process including register information and process memory maps (11.04 KB, application/octet-stream)
2005-02-07 08:07 UTC, Gregory G
no flags Details
Strace, debugger and maps with MALLOC_CHECK_=3 (19.54 KB, application/octet-stream)
2005-02-07 16:52 UTC, Gregory G
no flags Details

Description Paradigm Geophysical 2005-02-02 20:19:45 UTC
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

Description of problem:
The exec-shield functionality appears to cause our applications to 
crash randomly. When we disable the feature our programs work OK. 
The applications are technical applications for the oil
and gas industry. They make extensive use of shared objects.

Version-Release number of selected component (if applicable):
RHEE3U3 x86 : Linux hpnode4 2.4.21-20.ELsmp #1 SMP Wed Aug 18 
20:46:40 EDT 2004 i686 i686 i386 GNU/Linux

How reproducible:
Sometimes

Steps to Reproduce:
1. When our applications in user space with Exec-shield turned on
they experience random crashes. This goes away with the feature 
disabled
We would need to supply our apps to demonstrate.
    

Actual Results:  segmentation voilation

Expected Results:  normal completion

Additional info:

None

Comment 1 Michael Waite 2005-02-02 20:21:51 UTC
Verified as important to our customers and ISVs.

Comment 2 Arjan van de Ven 2005-02-02 20:23:58 UTC
what is the stack rlimit set to?

Comment 3 Arjan van de Ven 2005-02-02 20:26:27 UTC
also do you JIT code yourself? 

Comment 4 Michael Waite 2005-02-02 20:35:00 UTC
Alistair is the Emgineering Manager. The developers are in Isreal and
I will be adding them to this thread once I get their contact information.


Comment 5 Rik van Riel 2005-02-02 22:33:51 UTC
AFAIK, Exec-Shield is only turned on if the executable is marked with
a special ELF flag (IIRC PT_GNU_STACK).  For executables without that
flag set, the execshield feature should be disabled.

However, I do not know if the VM layout is also automatically
disabled. The answer to Arjan's stack rlimit question may help us
narrow things down further.

Comment 6 Gregory G 2005-02-04 09:46:12 UTC
gregoryg@p4-3 ~--> limit 
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       10240 kbytes
coredumpsize    unlimited
memoryuse       unlimited
vmemoryuse      unlimited
descriptors     1024 
memorylocked    4 kbytes
maxproc         7168 

We do have a code in our s/w that resets the stack limit to 8192KB, however, we 
have tried to remove it and play with the stack limits a little bit.

I would like to send you strace output with and without exec-shield enabled. 
How can I do it?


Comment 7 Michael Waite 2005-02-04 15:30:10 UTC
Just attach it to your next bugzilla entry.
"Create a new attachment" is the function you use to create and send an attachment.


Comment 8 Gregory G 2005-02-04 16:17:09 UTC
Created attachment 110657 [details]
Contains strace output with and without exec-shield option enabled

What information do you need us to provide you with?

Comment 9 Arjan van de Ven 2005-02-04 16:19:18 UTC
one of the key things we'd like to see is the /proc/<pid>/maps file at the time
of the crash. Normally that goes away immediately at crash time, however if you
run the app under gdb at crash time the file will stick around until gdb exists.
Also together with *the same* maps file, we'd like to know the crash address (eg
the EIP of the crash in the app) and in addition the stack pointer (ESP)

Comment 10 Gregory G 2005-02-07 08:07:23 UTC
Created attachment 110711 [details]
Debugging trace for the crashed process including register information and process memory maps

As per your request, the 19860.tgz attachement contains the process memory maps
together with a debugging session including backtrace for all threads on the
moment of crash and all registers.

We have means for remote debugging? Is it an option for you?

Thanks 
Giga

Comment 11 Gregory G 2005-02-07 16:52:55 UTC
Created attachment 110728 [details]
Strace, debugger and maps with MALLOC_CHECK_=3

We have discovered that the program crashes at much earlier stage when using
GLIBC malloc debugging hooks (MALLOC_CHECK_=3). 
Please, find strace output together with a debugging session and process memory
maps on the time of crash with malloc debugging enabled.

Thanks
Giga

Comment 12 RHEL Program Management 2007-10-19 19:07:50 UTC
This bug is filed against RHEL 3, which is in maintenance phase.
During the maintenance phase, only security errata and select mission
critical bug fixes will be released for enterprise products. Since
this bug does not meet that criteria, it is now being closed.
 
For more information of the RHEL errata support policy, please visit:
http://www.redhat.com/security/updates/errata/
 
If you feel this bug is indeed mission critical, please contact your
support representative. You may be asked to provide detailed
information on how this bug is affecting you.


Note You need to log in before you can comment on or make changes to this bug.