Bug 227735

Summary: Is it possible to make RHEL5 user friendly to start Sybase ASE?
Product: Red Hat Enterprise Linux 5 Reporter: Humphrey Lam <hlam>
Component: glibcAssignee: Jakub Jelinek <jakub>
Status: CLOSED NOTABUG QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.0CC: rdoty
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-02-07 23:32:48 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 219618    

Description Humphrey Lam 2007-02-07 20:55:48 UTC
Description of problem:
Sybase ASE 15.0 and 12.5.4 fails to start in RHEL5 in a default environment 
setting.   This problem is avoided by defining environment variable 
LD_POINTER_GUARD and set to 0.
    export LD_POINTER_GUARD=0

This deviates from RHEL3 and RHEL4.   Is it possible to make RHEL5 user 
friendly to start Sybase ASE by not having to define LD_POINTER_GUARD in the 
user environment?

Version-Release number of selected component (if applicable):


How reproducible:
Start Sybase ASE without defining LD_POINTER_GUARD.   This will mess up the 
setjmp() and longjmp() call in the ASE.

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Jakub Jelinek 2007-02-07 20:59:26 UTC
*** Bug 227736 has been marked as a duplicate of this bug. ***

Comment 2 Jakub Jelinek 2007-02-07 23:32:48 UTC
That means Sybase is broken, no applications should play with the implementation
details of jmp_buf.  Mangling jmp_buf in RHEL5+ is intentional, it is one of the
many added security measures.  So the answer to your question is no, it is not
desirable to change the default in RHEL5.  Either Sybase is fixed, or will need
the LD_POINTER_GUARD=0 workaround.

Comment 3 Humphrey Lam 2007-02-08 01:24:05 UTC
Please think about application compatibility.   When RHEL5 is released, our 
application becomes very user unfriendly to use by our customers.  Changing the 
environment variable (LD_POINTER_GUARD) is not straight forward in some of our 
bundling application like installer and others.  Not only do we need to change 
the script to set the environment LD_POINTER_GUARD but also we need to change 
some of our bundling application.  For example, our existing installer program 
when it is started as a parent application may not pass the environment setting 
to the child application.  Consequently the child application will not get the 
LD_POINTER_GUARD environment from the parent and will cause the trouble.  Our 
existing customer cannot simply update our application without we modify our 
application to suite your new OS.

We have a few questions:
1. Would it be possible to add a system tuning option using /sbin/sysctl to 
control this?
   This option is a compromise to us because we can ask our customers to set 
this option before installing our application.   This also bypasses the trouble 
to have we change our application to suite the new RHEL5.   We would be 
delighted to hear any other alternative way to make RHEL5 user friendly for 
running user application.

2. Could you provide us documentation as not to mangle jmp_buf?