Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 3 product line. The current stable release is 3.9. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 144299

Summary: setrlimit( RLIMIT_STACK, ... ) sets an unstable stack limit
Product: Red Hat Enterprise Linux 3 Reporter: Darin Ohashi <darin242>
Component: kernelAssignee: Jim Paradis <jparadis>
Status: CLOSED DUPLICATE QA Contact: Brian Brock <bbrock>
Severity: high Docs Contact:
Priority: medium    
Version: 3.0CC: peterm, petrides, riel
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-01-05 21:02:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
code to reproduce none

Description Darin Ohashi 2005-01-05 19:08:06 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5)
Gecko/20041107 Firefox/1.0

Description of problem:

Setting a stack limit using setrlimit( RLIMIT_STACK, ... ) causes the
actual stack limit to be very unstable and generally much smaller than
the desired value.

In the attached example, the stack limit is set to 100*1024 bytes,
however when executing the example, the segfault (stack limit
exceeded) occurs when only about 3000-6000 bytes of stack have been used.

I am running on an EM64T machine.  The same code executed on an
opteron running suse linux runs fine.


Version-Release number of selected component (if applicable):
kernel-2.4.21-20.0.1.EL

How reproducible:
Always

Steps to Reproduce:
1.  Compile the attached code.
2.  Run the resulting binary multiple times.
    

Actual Results:  em64t> repeat 10 ./a.out
11 = 3624, 142
11 = 5952, 239
11 = 5568, 223
11 = 5136, 205
11 = 5616, 225
11 = 5352, 214
11 = 5664, 227
11 = 6432, 259
11 = 4200, 166
11 = 5016, 200


Expected Results:  opteron> repeat 10 ./a.out
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111
11 = 98880, 4111

Additional info:

Comment 1 Darin Ohashi 2005-01-05 19:10:07 UTC
Created attachment 109388 [details]
code to reproduce

Comment 2 Arjan van de Ven 2005-01-05 19:13:35 UTC
to double check; this is with a 64 bit kernel and app, right?

Comment 3 Darin Ohashi 2005-01-05 19:18:10 UTC
Yes.

Comment 4 Ernie Petrides 2005-01-05 21:02:56 UTC
An errata has been issued which should help the problem 
described in this bug report. This report is therefore being 
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, 
please follow the link below. You may reopen this bug report 
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2004-550.html


*** This bug has been marked as a duplicate of 126297 ***