Bug 112559 - IBM java freezes in a few hours. has to kill process with -9
IBM java freezes in a few hours. has to kill process with -9
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: IBMJava2-JRE (Show other bugs)
3.0
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Karen Bennet
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2003-12-22 16:16 EST by chatpert
Modified: 2007-11-30 17:06 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2004-09-20 10:35:16 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description chatpert 2003-12-22 16:16:06 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3.1)
Gecko/20030425

Description of problem:
we are running an IM-type(instant messaging) server in a single process.
It was running fine on Redhat 9 with Sun java. After we upgraded to
redhat enterprise 3, we tried to use IBM java, but it hangs in a few
hours. Normal "kill" cannot kill the process, and we have to use "kill
-9". We gave up on IBM java and the info below refers to Sun java.

Then we switched back to Sun Java which was used redhat9. The problem
now is excessive memory usage. 
the java process's memory usage grows every day, now already over 1G
bytes.(in a week or so)
and it has not broken yet, but at some point it will.
This was not the case on redhat 9, and the memory usage was under 100M.
The system now is:
Linux 2.4.21-4.ELsmp #1 SMP i686 i686 i386 GNU/Linux
glib-1.2.10-11.1
glibc-2.3.2-95.6

We tried 2 java versions, and both have the same memory issue.
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Server VM (build 1.4.1_01-b01, mixed mode)

java version "1.4.2_03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_03-b02)
Java HotSpot(TM) Server VM (build 1.4.2_03-b02, mixed mode)

$JAVA -server -showversion

We want to downgrade to redhat 9. Is it possible to install redhat 9
kernel RPM on enterprise 3? Or change some config so that enterprise 3
uses redhat 9 kernel/library



Version-Release number of selected component (if applicable):
IBMJava2-JRE-1.4.1-8.i386.rpm

How reproducible:
Always

Steps to Reproduce:
1. Run a java server with a lot of thread creation and destrction.


    

Actual Results:  java process keeps using more and more memory.

Expected Results:  java process memory usage stablizes.

Additional info:

Because of this issue, we stopped our process of upgrading more redhat9
servers to Enterprise 3.
We think the problem has something to do with threads. Our java
program creates and destroys threads constantly, but at any given
time, there are less than 20 threads. Once we disable thread creation,
the memory usage stablizes.
Our guess is there is memory leak in the new thread library. Although
the java process uses more and more memory, the memory usage reported by
Runtime.getRuntime().totalMemory()
is very stable, under 70M.
Comment 1 chatpert 2003-12-23 20:07:44 EST
I found out this is a bug in java 1.4.
http://developer.java.sun.com/developer/bugParade/bugs/4959566.html

more from this page:



set environment variable LD_ASSUME_KERNEL=2.4.1, this will disable
NPTL and
avoid the memory leak.

xxxxx@xxxxx 2003-12-04

Evaluation 	

The errorid points to os_linux_i486.cpp, 774; further investigation shows
that a pthread_getattr_np() call just failed with ENOMEM. JVM relies on
pthread_getattr_np(), so it has no choice but to abort if the call fails.

Starting from Redhat AS-3 (NPTL-0.60 and later), pthread_getattr_np() will
allocate memory through malloc(), the memory is freed by
pthread_attr_destroy().
In JVM, we didn't call pthread_attr_destroy() after
pthread_getattr_np(), this
causes a small memory leak whenever pthread_getattr_np() is called (it is
called during thread creation).

In a long running application, especially when it has created lots of
threads, the memory leaks could add up and cause an out-of-memory error.
A temporary workaround is to disable NPTL by setting LD_ASSUME_KERNEL to
2.4.1.

xxxxx@xxxxx 2003-12-04

Fix is to call pthread_attr_destroy() after pthread_getattr_np(). It has
been verified with 1.4.2_02 JDK.

xxxxx@xxxxx 2003-12-08

Also I've been told that the issue has been fixed on the libc side too. It
will be available in the first update RHEL3 U1.

xxxxx@xxxxx 2003-12-08
Comment 2 Thomas Fitzsimmons 2004-09-20 10:35:16 EDT
Fixed in RHEL3 U2.  Closing.

Note You need to log in before you can comment on or make changes to this bug.