Description of problem: When attempting to upload a WAR that is ~60MB using the CLI, it fails with the following error: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2798) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:111) at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:78) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1857) at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1766) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:346) at org.jboss.remoting.serialization.impl.java.JavaSerializationManager.sendObjectVersion2_2(JavaSerializationManager.java:120) at org.jboss.remoting.serialization.impl.java.JavaSerializationManager.sendObject(JavaSerializationManager.java:95) at org.jboss.remoting.marshal.serializable.SerializableMarshaller.write(SerializableMarshaller.java:120) at org.jboss.remoting.marshal.http.HTTPMarshaller.write(HTTPMarshaller.java:73) at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:279) at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:137) at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122) at org.jboss.remoting.Client.invoke(Client.java:1634) at org.jboss.remoting.Client.invoke(Client.java:548) at org.jboss.remoting.Client.invoke(Client.java:536) at org.rhq.enterprise.clientapi.RemoteClientProxy.doInvoke(RemoteClientProxy.java:85) at org.rhq.bindings.client.AbstractRhqFacadeProxy.invoke(AbstractRhqFacadeProxy.java:87) at org.rhq.enterprise.clientapi.RemoteClientProxy.invoke(RemoteClientProxy.java:69) at $Proxy14.createPackageVersionWithDisplayVersion(Unknown Source) at org.rhq.bindings.client.ResourceClientProxy$ClientProxyMethodHandler.updateBackingContent(ResourceClientProxy.java:545) at org.rhq.bindings.client.ResourceClientProxy$ClientProxyMethodHandler.updateBackingContent(ResourceClientProxy.java:519) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.rhq.bindings.client.ResourceClientProxy$ClientProxyMethodHandler.invoke(ResourceClientProxy.java:602) at org.rhq.bindings.client.ResourceClientProxy_$$_javassist_0.updateBackingContent(ResourceClientProxy_$$_javassist_0.java) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Version-Release number of selected component (if applicable): 4.4.0.JON312GA How reproducible: Always Steps to Reproduce: 1. Install JBoss ON remote API and CLI (rhq-remoting-cli). 2. Copy example varia/threaddump.war from JBoss EAP 5 to tmp: cp -a "${JBOSS_HOME}/docs/examples/varia/threaddump.war" /tmp 3. Create a file with random contents to create WAR bloat to equal ~60MB: dd if=/dev/urandom of=/tmp/random-junk.file bs=1024K count=60 4. Add random-junk.file to jboss-as-helloworld.war: cd /tmp zip threaddump.war random-junk.file 5. Create JBoss ON CLI test script: cat >/tmp/upload-large-war-test.js <<EOF var criteria = new ResourceCriteria(); var resource = new Resource(); criteria.addFilterName("threaddump.war"); criteria.addFilterResourceTypeName('Web Application (WAR)'); criteria.addFilterPluginName('JBossAS'); resource = ResourceManager.findResourcesByCriteria(criteria).get(0); var managed_resource = ProxyFactory.getResource(resource.id); while (managed_resource.getBackingContent() == null) { // We have to do this because it take some time for the backing content to actually show up. java.lang.Thread.currentThread().sleep(5000); managed_resource = ProxyFactory.getResource(resource.id); // We seep again because it seems that the proxy factory may not have fully constructed the object in time java.lang.Thread.currentThread().sleep(1000); } managed_resource.updateBackingContent("/tmp/threaddump.war"); EOF 6. Copy threaddump.war to 'all' deploy directory: cp -a /tmp/threaddump.war ${JBOSS_HOME}/server/all/deploy 7. Start EAP 5.2 'all' server. 8. Start JBoss ON system. 9. Import EAP server into inventory. 10. Execute CLI test script: ./rhq-cli.sh -u rhqadmin -p rhqadmin -s localhost -t 7080 -f /tmp/upload-large-war-test.js Wait sever minutes as it will take the conent a while to make its way into the database. Actual results: Execution will fail with the following error: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2798) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:111) at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:78) Expected results: Execution should not fail and content should be updated. Additional info:
Fixed in master commit 0b99feed5411fe1b3437f9e64953c4818a5a4289 Author: Thomas Segismont <tsegismo> Date: Tue Sep 10 10:15:52 2013 +0200 The CLI uses JBoss Remoting to communicate with the server. JBoss Remoting relies on JDK's HttpUrlConnection when configured for the HTTP transport. But HttpUrlConnection copies the full payload of the HTTP request in memory before sending it to the server. Therefore, when you send a huge file, you can easily reach the maximum heap. Another problem was that the CLI itself was loading the file in memory as well, just to compute the file checksum. Updated ResourceClientProxy to compute checksum on FileInputStream instead of file bytes. Created new methods in ContentManager to upload content in fragments, updated ResourceClientProxy accordingly.
verified on JON 3.2.0.ER2