Bug 829399

Summary: Large files deployed or managed in content system (WAR/EAR/bundle/etc) can cause OutOfMemory failures
Product: [JBoss] JBoss Operations Network Reporter: Armine Hovsepyan <ahovsepy>
Component: ContentAssignee: Thomas Segismont <tsegismo>
Status: CLOSED DUPLICATE QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: unspecified    
Version: JON 3.1.0CC: hrupp, loleary, mfoley, tsegismo
Target Milestone: ---   
Target Release: JON 3.2.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 879051 (view as bug list) Environment:
Last Closed: 2013-06-13 14:43:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 879051    
Bug Blocks:    
Attachments:
Description Flags
error log
none
error_eap5.log none

Description Armine Hovsepyan 2012-06-06 15:28:37 UTC
Created attachment 589943 [details]
error log

Description of problem:
While trying to deploy a large war (>250 MB) under EAP6 via JON_GUI, it shows that deployment went OK on GUI, while there is a java heap space - OutOfMemory exception in server log.

Version-Release number of selected component (if applicable):
JON - Version: 3.1.0.ER6

How reproducible:
always

Steps to Reproduce:
1. Create child resource under EAP6 as Deployment resource
2. Upload a big war (> 250 MB)
3. Click on Next -> Finish
  
Actual results:
Big war should have been deployed under EAP6

Expected results:
War is not being deployed under EAP and green message about deployment is being shown. (java heap space error apears.)

Additional info:
Please get attached the log.
java heap space params - "-Xms1024M -Xmx1024M -XX:PermSize=256M -XX:MaxPermSize=256M -Djava.net.preferIPv4Stack=true"

Comment 1 Armine Hovsepyan 2012-06-06 15:44:36 UTC
Hi all,

Please ask me for the large war, haven't yet found the best way/place to keep it.

Regards,
Armine H

Comment 2 Heiko W. Rupp 2012-06-08 12:20:17 UTC
Have you tried this with any other plugin too?
What you describe sounds like a out of memory in the server and seems to have 
nothing to do with the As7-plugin.

Comment 3 Stefan Negrea 2012-08-07 22:15:08 UTC
Please retest the same scenario but with the AS5 or AS4 plugin. As Heiko commented, the issues seems to be related to the content system and not AS7 plugin in particular.

Comment 4 Armine Hovsepyan 2012-08-09 09:39:54 UTC
Hi Stefan,

I've checked with EAP5 as well and it's getting java.lang.InterruptedException. Please get attached log.

Regards,
Armine H

Comment 5 Armine Hovsepyan 2012-08-09 09:40:38 UTC
Created attachment 603219 [details]
error_eap5.log

Comment 6 Armine Hovsepyan 2012-08-09 11:20:17 UTC
Putting this back to ON_DEV, and later we could change either the title (like EAP6) or close and file another bug.

Comment 7 Stefan Negrea 2012-08-14 17:09:00 UTC
This is a general problem related to memory allocation. The errors occur because a large file is stored in memory during the deployment process. While the problem was spotted so far on the server that is not the only place where it can occur. 

The life-cycle of a file:
1) User uploads the file on the server
2) The server stores the file in the database
3) The server sends the file to the agent
4) The agent delivers the file to the managed resource

This process can fail at any point due to running out of memory. Even if the server has enough memory available, that might not be the case for the agent, the managed resource, or even the database itself. The point of failure is unpredictable and it all depends on the configuration and current state of each component of this deployment process.

This a design and implementation problem with the current content system. While there are some better techniques to deal with such large content, the current implementation of the content system uses a memory intensive solution. This is not a regression. As seen so far, the content system works with moderately sized files.

Comment 8 Charles Crouch 2012-08-29 16:07:31 UTC
As per 8/20 triage 8/20 pushing to JON312

Comment 10 Mike Foley 2012-10-18 19:45:54 UTC
if deployment fails ...for whatever reason the JON UI should convey this to the user ... 

JON should display information in the JON UI (not in the server log ... not in the agent log) that a failure occured.

the actual out-of-memory issue ... that is another issue.

Comment 11 Mike Foley 2012-10-18 19:46:20 UTC
if deployment fails ...for whatever reason the JON UI should convey this to the user ... 

JON should display information in the JON UI (not in the server log ... not in the agent log) that a failure occured.

the actual out-of-memory issue ... that is another issue.

Comment 12 mark yarborough 2012-11-20 20:45:28 UTC
Per triage with loleary, crouch, mfoley: Move to JBoss ON product, set target release JON 3.2, clear priority (will be subject to further triage in JON 3.2 timeframe).

Comment 13 Thomas Segismont 2013-06-13 14:43:38 UTC
Closing as a duplicate of BZ887861

Tested with a 254 Mo WAR on AS4.2.3, EAP5.1 and EAP6.1

*** This bug has been marked as a duplicate of bug 887861 ***