Back-port to JBoss ON 3.1.2 as the current implementation used results in connection management/caching issues that can not be fixed due to the underlying implementation in the JVM. This fix is necessary to resolve other AS7 plug-in issues in JBoss ON 3.1.2. For example Bug 918677 - [as7] ASConnection becomes invalid and stale if security realm is reset or has a partition event resulting in loss of management and monitoring of AS7 resources +++ This bug was initially created as a clone of Bug #950660 +++ +++ This bug was initially created as a clone of Bug #929200 +++ +++ This bug was initially created as a clone of Bug #929198 +++ In AS7 plugin, ASConnection class currently uses JDK's HttpUrlConnection to communicate with an AS7 instance. This class should be updated to use Commons HttpClient, like ASUploadConnection class does since release 4.6 (see BZ887320) Commons HttpClient gives us more control on the communication process and has proven to be more reliable. --- Additional comment from Thomas Segismont on 2013-03-29 14:28:38 CET --- See BZ918677 for the type of problems we have with JDK's HttpUrlConnection --- Additional comment from Charles Crouch on 2013-03-29 15:34:29 CET --- Since this seems to be related more to fixing a bug than adding a feature I've removed the FutureFeature keyword. I've also unset the Target Release field to make sure this gets triaged. A change in the http library used by the AS7 plugin will require a full regression test run, since http communication is used so widely to talk to AS7 --- Additional comment from Thomas Segismont on 2013-04-02 10:57:36 CEST --- BZ918677 has been partially fixed (only works on some platforms). Moving ASConnection to Commons HttpClient will allow to get rid of the problem on all platforms. ASConnection is the core communication component of AS7 plugin. Changing its implementation will involve extensive non regression testing from QE before the next JON release. But as ASUploadConnection has already been moved to commons HttpClient due to BZ887320, it has to be done anyway. As for the benefits of this change: * cohesion between ASConnection and ASUploadConnection implementations * easier debugging as JDK's HttpUrlConnection is closed source (at least in Oracle's VM) * Commons HttpClient gives the developer more control on the communication process (when/how open or close connections, configurable pooling, unshared authentication) --- Additional comment from Thomas Segismont on 2013-04-02 11:56:28 CEST --- And for the risks. No instance of HttpUrlConnection is shared outside ASConnection. So components in AS7 plugin only rely on ASConnection contract. Consequently, while ASConnection is the core communication component of AS7 plugin, the risk of regression is very low. --- Additional comment from Thomas Segismont on 2013-04-08 23:42:52 CEST --- Fixed in master - 21ec5e2 AS7 plugin itests passed succesfully Beware that this touches the core connection part (ASConnection class) and hence requires a full non regresssion testing of AS7 plugin. --- Additional comment from Thomas Segismont on 2013-04-09 12:08:13 CEST --- Reworked to increased delay in AS7 itests to discover all resources master - 3b18013 --- Additional comment from Thomas Segismont on 2013-04-10 11:17:22 EDT --- Changes reported from master to release/jon3.1.x - 89d941d All unit and integration tests pass on my box and on Jenkins: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/RHQ/job/rhq-master/org.rhq$rhq-jboss-as-7-plugin/
Needs verified from hotfix patch based on build org.jboss.on-jboss-on-parent-3.1.2.GA-11[1]. [1]: https://brewweb.devel.redhat.com//buildinfo?buildID=266751
The as-7 plug-in from build org.jboss.on-jboss-on-parent-3.1.2.GA-11 is contained as attachment 736012 [details] of Bug 952330 - EAP Plugin Pack Hotfix-02 for JBoss ON 3.1.2
Above patch causes regression. I am randomly getting this error on various stuff I do with EAP 6.0 server java.lang.Exception: The target server failed to respond Reload operation, Creating/Deleting resources,deployments Another thing I observe: it seems that new http connection instance cannot be shared among threads. I run "discovery -f" on agent. While it's running, I run "reload" operation on AS7 .. such operation seems to be blocked for a long time (usually it takes abou 5-10 seconds). In this case it takes about 2 minutes and then it fails with "The target server failed to respond"
(In reply to comment #4) > Above patch causes regression. > > I am randomly getting this error on various stuff I do with EAP 6.0 server > > java.lang.Exception: The target server failed to respond This indeed may come from HttpClient library. I'll have to check why it happens. > > Reload operation, Creating/Deleting resources,deployments Do you mean you got the previous exception while doing these actions? > > Another thing I observe: it seems that new http connection instance cannot > be shared among threads. > > I run "discovery -f" on agent. While it's running, I run "reload" operation > on AS7 .. such operation seems to be blocked for a long time (usually it > takes abou 5-10 seconds). In this case it takes about 2 minutes and then it > fails with "The target server failed to respond" How can I reproduce this? On my box, "discovery -f" completes before I can click the UI.
Partially fixed in master - 915ab3d (no more "The target server failed to respond" errors). Keepalive timeout paramater was too high. The server could close connection server-side and the agent would still try to reuse it. Now HttpClient uses the same value as HttpUrlConnection did to expire connections (5 seconds if the server does not send its keepalive timeout in reponse headers). Deployments should just work as they don't use persistent connections from ASConnection class. My testing confirms that. As for the reload operation problem, the operation will fail even if you don't trigger a full discovery on the agent at the same time. I tried the same steps with unpatched 3.1.2 build and it also fails. The message is just different. Working on a complementary fix.
Pushed commit 1c01db5 to release/jon3.1.x branch (cherry-picked from master - 915ab3d) as integration tests passed: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/RHQ/job/rhq-master/org.rhq$rhq-jboss-as-7-plugin/2168/
Fixed in EAP Plugin Pack Hotfix-02 for JBoss ON 3.1.2.