Bug 747243

Description of problem:
I’m hoping you can help address some serious issues we are experiencing with our latest product offer when deployed on Red Hat Enterprise Linux 6 Update 1 (64 bit). 

The issue wasn’t discovered during functional testing, appears to be load related and cannot be recreated if we perform the same load test using Red Hat Enterprise Linux Version 5 Update 6 (64 bit). 

Right now this is a major blocker to our release schedule and is impacting customers wishing to deploy our product on the latest Red Hat release. 

Background: we run a large heavily multi-threaded java application as a non-privileged user. It uses up to 500MB memory, several thousand sockets and file handles, and many hundreds of threads. Basically we launch the application using an /etc/init.d script which boils down to the following: 

ulimit -n 8192 
ulimit -u 2048 
su - nonprivuser -c "cd /somedirectory; ./wrapper64 ..." 

wrapper64 is a java launcher application that launches the java virtual machine as a second process - we use the industry standard tanukisoft one. The wrapper launches the java vm with default command line options except for -Xmx512m to increase the max allowed heap to half a gig. 

Issue 1 (major) - after a few tens of seconds of starting up the application has started creating many of its sockets and file handles and started allocating memory as the heap grows. This seems to exhaust some resource for the whole of the "nonprivuser" user. It becomes impossible to create new processes as this user (i.e. cannot log on, cannot su to this user, cannot run any commands as this user (bash cannot fork)) - forks in particular seem to fail with EAGAIN). This does not happen in RHEL 5. 

Issue 2 (minor) - in RHEL 5 we do not need to include the line ulimit -u 2048 above. In RHEL 6, without it, the java application is unable to create more than a few hundred threads. The other issue we have with "-u" is that the soft limit doesn't seem to persist across su. 
I.E.
root# ulimit -u 2048 
root# su - nonprivuser 
nonpriv$ ulimit -u 

gives 1024 (although nonpriv seems to be able to ulimit -u 2048 successfully) 
Why is it necessary to invoke ulimit -u twice?


Version-Release number of selected component (if applicable):


How reproducible:
Consistently under load

Steps to Reproduce: 
See description above
  
Actual results:
See description above


Expected results:
See description above


Additional info: