Workaround: Workaround Exists Workaround Description: Use a different VFSCache implementation, or set the default lifetime to something really high. JAVA_OPTS="$JAVA_OPTS -Djboss.vfs.cache=org.jboss.virtual.plugins.cache.IterableTimedCache -Djboss.vfs.cache.TimedPolicyCaching.lifetime=140000" project_key: SOA There's a problem with deployment of esb application, where the camel gateway is used . If is the application deployed right after the server start, there's no problem. But if you wait about 30-60 minutes, error occurs, and about 100 of this error message are shown: DEBUG [JBossPackageScanClassResolver] Cannot find class 'apache/camel/converter/stream/InputStreamCache.class' in any classloaders: [BaseClassLoader@2444569{vfszip:/home/rbalent/qa/tests/quickstarts/tests/output/lib/Quickstart_camel_http.esb/}] To see the messages you must set log threshold to DEBUG. To reproduce it, just start server, deploy attached Quickstart_camel_http.esb, then you will see the correct message in server log: BEFORE** Camel HTTP Hello World! AFTER** Lines "BEFORE**", and "AFTER**" are added to find the message body in server log easier. then undeploy Quickstart_camel_http.esb and wait about 30-60 minutes. After this time, deploy the esb package again. You will see the mentioned lines in server log, and incorrect message body: The received message will look like this: BEFORE** java.io.ByteArrayInputStream@6c515139AFTER** After server restart it works again. Quickstart_camel_http.esb, server log of successful deployment and failed deployment are attached.
files attached
Attachment: Added: Quickstart_camel_http.esb Attachment: Added: failed_deployment.log Attachment: Added: successful_deployment.log
This sounds suspiciously like a classloader leak, will do some investigation over the next day or so.
Link: Added: This issue is related to SOA-3210
Link: Removed: This issue is related to SOA-3210
This seems related to JBVFS-134. Notice in the bug description in the DEBUG message that the first part of the package name of the classes are truncated. That's happening because IterableTimedVFSCache.findContext(uri) is returning a different context after the 30 minutes (the default lifetime of the cache). The incorrect context affects the way the class name is parsed, and we end up with the incorrectly parsed classnames which won't resolve with any classloader because they are wrong. I am not sure whether the code creating the context is wrong, or whether there's a ClassLoader leak here which is making it behave incorrectly - if I have time I'll do some additional research into that. There is an easy workaround here though - just use a different VFSCache implementation. Another thing that should be noted is that the .esb archive posted here goes against the directions on using third party Camel components on the Wiki. The wiki article specifies that these should be installed into esb.deployer/lib rather than within the ESB archive.
Link: Added: This issue is related to JBVFS-134
Workaround Description: Added: Use a different VFSCache implementation, or set the default lifetime to something really high. JAVA_OPTS="$JAVA_OPTS -Djboss.vfs.cache=org.jboss.virtual.plugins.cache.IterableTimedCache -Djboss.vfs.cache.TimedPolicyCaching.lifetime=140000" Workaround: Added: [Workaround Exists]
Oct 4, 2011 bug triage - move Fix Version to Future
Workaround exists, should follow directions for third party Camel components. Closing as won't fix.
Two questions: 1. Does this occur only after undeploy or even on initial deployment after server idle? 2. What are the (side) effects of the workaround approaches for other components/deployments?
1. Yes, it occurs for initial deployment too. Just run server, wait jboss.vfs.cache.TimedPolicyCaching.lifetime and hotdeploy application.
I'm not sure if "VFS cache change" workaround works correctly. Can somebody try to reproduce it with me? Reproducing the original reported bug: 1. Copy camel-http-2.4.0.jar to deployers/esb.deployer/lib as Tom recommended. 2. Set logging same as in all configuration to see output. 3. Run server with -Djboss.vfs.cache.TimedPolicyCaching.lifetime=180 so you can easily wait for cache to expire. 4. Wait 3 minutes for cache expiration 5. Copy New_Quickstart_camel_http.esb to deploy folder. You will see in server log: BEFORE** java.io.ByteArrayInputStream@6c515139AFTER** 6. Correct output is: BEFORE** Camel HTTP Hello World! AFTER** Try same with the 3 minutes cache expiration and append -Djboss.vfs.cache=org.jboss.virtual.plugins.cache.IterableTimedCache You will probably see same incorrect output after 3 minutes. If you deploy esb application before 3 minutes expires, You will see the correct output. Will attach camel-http-2.4.0.jar and New_Quickstart_camel_http.esb files.
Created attachment 568948 [details] camel-http-2.4.0.jar
Created attachment 568950 [details] New_Quickstart_camel_http.esb
I've created JBPAPP-8411 issue in jira.
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: If you deploy an ESB application with a non-core Camel gateway, it will stop working after about thirty minutes. After this time, an error is thrown. To work around this, deploy your application before this limit or raise the jboss.vfs.cache.TimedPolicyCaching.lifetime property to a higher value.
That description isn't really accurate. It should read something like : There is an issue with VFS caching and the deployment of non-core Camel components after a server has been booted/started and running for more than thirty minutes. No issue is present if you deploy a non-core Camel component before boot. After the thirty minutes, if you deploy a non-core Camel component, an error is thrown. To work around this, deploy your application before this 30 minute limit or raise the jboss.vfs.cache.TimedPolicyCaching.lifetime property to a higher value.
Take 2 - I think this is even a little more clear : There is an issue with VFS caching and the deployment of non-core Camel components in the a situation where the server has been booted/started and running for more than thirty minutes. No issue is present if you deploy a non-core Camel component before booting the server, or before the cache lifetime expires (30 minutes). After the cache lifetime expires (30 minutes), if you deploy a non-core Camel component, an error is thrown. To work around this, deploy your application before starting the server or before this 30 minute limit or raise the jboss.vfs.cache.TimedPolicyCaching.lifetime property to a higher value.
Thanks very much for that Tom. Have altered the note.
Technical note updated. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1 +1 @@ -If you deploy an ESB application with a non-core Camel gateway, it will stop working after about thirty minutes. After this time, an error is thrown. To work around this, deploy your application before this limit or raise the jboss.vfs.cache.TimedPolicyCaching.lifetime property to a higher value.+There is an issue with VFS caching and the deployment of non-core Camel components where the server has been running for more than thirty minutes. After the cache lifetime expires (30 minutes), if you deploy a non-core Camel component, an error is thrown. To work around this, deploy your application before starting the server or before this 30 minute limit, or raise the boss.vfs.cache.TimedPolicyCaching.lifetime property to a higher value. (No issue is present if you deploy a non-core Camel component before booting the server, or before the cache lifetime expires.)
Hello, I was just looking a little at this problem today and maybe I found the solution. In class: org.jboss.soa.esb.listeners.gateway.camel.JBossPackageScanClassResolver$MatchingClassVisitor method: "visit(VirtualFile file)" it could look like this. I compiled the class, tested and can't reproduce the problem anymore. Can somebody review my code? public void visit(VirtualFile file) { if (file.getName().endsWith(".class")) { String fqn; try { fqn = file.toURL().toString(); } catch (Exception ex) { fqn = file.getName(); } String qn; if (fqn.indexOf("jar/") != -1) { qn = fqn.substring(fqn.indexOf("jar/") + 4); } else { qn = fqn; } addIfMatching(filter, qn, classes); } }
Reviewed and tested Robert's patch - I don't see any errors with it using the attached quickstart. Committed to 4_11_CP branch.