Bug 134782 - /proc/sys/kernel/exec-shield-randomize=1 breaks programs that shmat()
Summary: /proc/sys/kernel/exec-shield-randomize=1 breaks programs that shmat()
Alias: None
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel
Version: 3.0
Hardware: i686
OS: Linux
Target Milestone: ---
Assignee: Ingo Molnar
QA Contact:
Depends On:
TreeView+ depends on / blocked
Reported: 2004-10-06 09:24 UTC by Ronald Cole
Modified: 2008-08-02 23:40 UTC (History)
4 users (show)

Clone Of:
Last Closed: 2007-10-19 19:17:01 UTC

Attachments (Terms of Use)

Description Ronald Cole 2004-10-06 09:24:08 UTC
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 
SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)

Description of problem:
Update 3 added a new feature "Exec Shield Randomize" and turned it on 
by default.  IBM Informix 4GL application compiled with this release 
will unpredictably fail to connect to a local IBM Informix database 
instance because shmat() will fail to attach shared memory at the 
configured location.  There doesn't appear to be a "fine-grained" way 
to turn off this random VM mapping per binary without either also 
disabling Exec Shield or turning random VM mapping off globally.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Compile an IBM Informix 4GL application with Update 3 compiler 
2. Sit back and wait for shmat() to fail at an inopportune moment.

Actual Results:  Angry 4GL application users/Puzzled 4GL application 

Expected Results:  shmat() not failing unpredictably.

Additional info:

Is there some clever way to use setarch to disable randomization 
without also disabling exec shield?  Or some utility to discern if 
any given binary will even employ Exec Shield?  The release notes are 
somewhat less than detailed.

Comment 1 Josh Bressers 2004-12-07 20:15:55 UTC
I'm removing the security severity on this issue.

Comment 2 Ingo Molnar 2005-09-15 17:57:59 UTC
calling it in a setarch -3 <command> wrapper should turn off randomization for
that app only. Alternatively, calling personality(PER_LINUX32) will turn off
randomization too (on x86).

Comment 3 Ronald Cole 2007-07-29 02:36:55 UTC
This bug can probably be closed.  Not only is it ancient, but I ultimately
solved it by lowering the address of the Informix shared memory segment that I
was attaching to.

Comment 4 RHEL Product and Program Management 2007-10-19 19:17:01 UTC
This bug is filed against RHEL 3, which is in maintenance phase.
During the maintenance phase, only security errata and select mission
critical bug fixes will be released for enterprise products. Since
this bug does not meet that criteria, it is now being closed.
For more information of the RHEL errata support policy, please visit:
If you feel this bug is indeed mission critical, please contact your
support representative. You may be asked to provide detailed
information on how this bug is affecting you.

Note You need to log in before you can comment on or make changes to this bug.