This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1459641 - HotSpot leaking memory in long-running requests
HotSpot leaking memory in long-running requests
Status: ON_QA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: java-1.8.0-openjdk (Show other bugs)
7.3
x86_64 Linux
medium Severity medium
: rc
: ---
Assigned To: Andrew John Hughes
BaseOS QE - Apps
:
Depends On:
Blocks: 1420851
  Show dependency treegraph
 
Reported: 2017-06-07 13:04 EDT by Deepu K S
Modified: 2017-10-20 07:07 EDT (History)
9 users (show)

See Also:
Fixed In Version: java-1.8.0-openjdk-1.8.0.141-3.b16.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Icedtea Bugzilla 3412 None None None 2017-06-27 22:06 EDT
openjdk bug system JDK-8164293 None None None 2017-06-07 13:09 EDT

  None (edit)
Description Deepu K S 2017-06-07 13:04:37 EDT
Description of problem:
HotSpot leaking memory in long-running requests. While checking the RSS part, it is seen memory usage is going high gradually with time.

# java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)


Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux 7.3
java-1.8.0-openjdk-1.8.0.131-3.b12.el7_3.x86_64

How reproducible:
Always.

Steps to Reproduce:
1. Check out the sample project from GitHub onto a RHEL 7 system.
2. cd into nacroleptic
3. Run # mvn clean package
4. Run java -jar target/narcoleptic.jar
5. cd into ../devourer
6. Run # mvn clean package
7. Run ./run.sh
8. Download Apache JMeter 3.0 from http://jmeter.apache.org/download_jmeter.cgi
Extract the archive and start with the "bin/jmeter.sh" script.
9. Open the file "load_test.jmx" with JMeter.
10. Start the JMeter tests with CTRL+R

Leave for *at least* 30 minutes, then come back and observe the amount of memory used by the process. It will have increased beyond what would be expected. Leave for several hours, maybe a couple of days. Memory usage will continue to increase.
Checked from RSS values under ps/top. 

Tried with running the application under valgrind --tool=massif, but the application gets killed after a few seconds.

Actual results:
Memory usage of the application increases over time, without stopping. It may slow, and *appear* to stop, but given a large enough time span, it will *always* continue to increase.

Expected results:
Memory usage of the application remains stable.

Additional info:
Upstream bugs :
https://bugs.openjdk.java.net/browse/JDK-8178124
https://bugs.openjdk.java.net/browse/JDK-8164293
Fix appears to be added to 8u152 and is backported to 8u131 b31. We need the fix backported to RHEL OpenJDK or an openjdk update.
Comment 2 Andrew John Hughes 2017-06-09 13:10:15 EDT
With this already being backported to 8u131 b31, I suspect it will form part of 8u141 next month.
Comment 3 Deepu K S 2017-07-14 07:18:22 EDT
(In reply to Andrew John Hughes from comment #2)
> With this already being backported to 8u131 b31, I suspect it will form part
> of 8u141 next month.

I see this is moved to RHEL 7.5
Does this mean this will not be fixed in the openjdk update coming with rhel-7.4?

Thanks.
Comment 4 Andrew John Hughes 2017-07-14 11:14:25 EDT
(In reply to Deepu K S from comment #3)
> (In reply to Andrew John Hughes from comment #2)
> > With this already being backported to 8u131 b31, I suspect it will form part
> > of 8u141 next month.
> 
> I see this is moved to RHEL 7.5
> Does this mean this will not be fixed in the openjdk update coming with
> rhel-7.4?
> 
> Thanks.

Yes, it was submitted too late to make rhel-7.4.

It also doesn't appear to be included in 8u141 as we've received it.

The fastest route to get a fix in 7.4 will be to request z-stream once this bug is resolved (waiting on PM approval; I'll ping them today)
Comment 6 Andrew John Hughes 2017-08-14 16:14:11 EDT
This is fixed in the upcoming 7.5. If it's still desired in 7.4, then now is the time to request it for z-stream.
Comment 8 Ding-Yi Chen 2017-10-11 03:10:14 EDT
According to https://bugs.openjdk.java.net/browse/JDK-8164293.
Isn't the build for  u144 and earlier is b32?
Comment 10 Andrew John Hughes 2017-10-11 12:06:19 EDT
(In reply to Ding-Yi Chen from comment #8)
> According to https://bugs.openjdk.java.net/browse/JDK-8164293.
> Isn't the build for  u144 and earlier is b32?

It's not present in 8u144.

Those b31/b32 fixes are misleading. They seem to be builds made for specific Oracle customers only, as far as I can gather.

The relevant bits are the fixed version (8u152) and the line for 8u161 b01.
8u161 is the January 2018 security update, which will be based on the 8u152 feature release expected next week. 

So, this should be fixed everywhere by the time of the January CPU. I expect us to try and get 8u152 into RHEL 7 some time in November.

Note You need to log in before you can comment on or make changes to this bug.