Bug 134782

Summary: /proc/sys/kernel/exec-shield-randomize=1 breaks programs that shmat()
Product: Red Hat Enterprise Linux 3 Reporter: Ronald Cole <ronald>
Component: kernelAssignee: Ingo Molnar <mingo>
Status: CLOSED WONTFIX QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0CC: barryn, petrides, riel, sandor.szabo
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-10-19 19:17:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ronald Cole 2004-10-06 09:24:08 UTC
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 
SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)

Description of problem:
Update 3 added a new feature "Exec Shield Randomize" and turned it on 
by default.  IBM Informix 4GL application compiled with this release 
will unpredictably fail to connect to a local IBM Informix database 
instance because shmat() will fail to attach shared memory at the 
configured location.  There doesn't appear to be a "fine-grained" way 
to turn off this random VM mapping per binary without either also 
disabling Exec Shield or turning random VM mapping off globally.


Version-Release number of selected component (if applicable):
2.4.21-20.EL

How reproducible:
Sometimes

Steps to Reproduce:
1. Compile an IBM Informix 4GL application with Update 3 compiler 
toolchain.
2. Sit back and wait for shmat() to fail at an inopportune moment.
3.
    

Actual Results:  Angry 4GL application users/Puzzled 4GL application 
developers.

Expected Results:  shmat() not failing unpredictably.

Additional info:

Is there some clever way to use setarch to disable randomization 
without also disabling exec shield?  Or some utility to discern if 
any given binary will even employ Exec Shield?  The release notes are 
somewhat less than detailed.

Comment 1 Josh Bressers 2004-12-07 20:15:55 UTC
I'm removing the security severity on this issue.

Comment 2 Ingo Molnar 2005-09-15 17:57:59 UTC
calling it in a setarch -3 <command> wrapper should turn off randomization for
that app only. Alternatively, calling personality(PER_LINUX32) will turn off
randomization too (on x86).

Comment 3 Ronald Cole 2007-07-29 02:36:55 UTC
This bug can probably be closed.  Not only is it ancient, but I ultimately
solved it by lowering the address of the Informix shared memory segment that I
was attaching to.

Comment 4 RHEL Program Management 2007-10-19 19:17:01 UTC
This bug is filed against RHEL 3, which is in maintenance phase.
During the maintenance phase, only security errata and select mission
critical bug fixes will be released for enterprise products. Since
this bug does not meet that criteria, it is now being closed.
 
For more information of the RHEL errata support policy, please visit:
http://www.redhat.com/security/updates/errata/
 
If you feel this bug is indeed mission critical, please contact your
support representative. You may be asked to provide detailed
information on how this bug is affecting you.