Bug 804102

Summary: [as7] start, shutdown, restart, and reload operations should block/poll until the underlying task has completed
Product: [Other] RHQ Project Reporter: Ian Springer <ian.springer>
Component: PluginsAssignee: Libor Zoubek <lzoubek>
Status: CLOSED CURRENTRELEASE QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: urgent    
Version: 4.3CC: ccrouch, hrupp, lzoubek, theute
Target Milestone: ---   
Target Release: RHQ 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=893802
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-09-01 06:04:39 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 707223    

Description Ian Springer 2012-03-16 10:42:47 EDT
This way the RHQ user knows that if the RHQ operation succeeded, they can be fairly confident the task did as well. 

The as4 and as5 plugins both worked this way. For example, in the as5 plugin, see ApplicationServerOperationsDelegate.waitForServerToStart() and ApplicationServerOperationsDelegate.waitForServerToShutdown().
Comment 1 Mike Foley 2012-03-19 11:34:53 EDT
per BZ triage (crouch, loleary, mfoley)
Comment 2 Heiko W. Rupp 2012-03-23 13:04:44 EDT
master 394af648a52f1a
Comment 3 Ian Springer 2012-03-23 16:56:00 EDT
This doesn't seem to be working quite right.

When I invoke the domain shutdown operation, I can see from the log that the server gets shutdown:

=== Invoking operation [shutdown] with parameters [null] on Resource[id=27, uuid=9bc273fb-3d79-4ae3-b887-83277850615e, type={jboss-as-7}JBossAS7 Host Controller, key=DomainController, name=DomainController, parent=pc, version=7.1.2.Final-SNAPSHOT]...
16:34:31,947 INFO  [org.jboss.as.process] (Thread-14) JBAS012016: Shutting down process controller
16:34:31,947 INFO  [org.jboss.as.process.Host Controller.status] (Thread-14) JBAS012018: Stopping process 'Host Controller'
16:34:31,963 ERROR [ASConnection] Failed to get data: Connection refused
16:34:31,966 ERROR [ASConnection] Failed to get data: Connection refused
16:34:31,966 DEBUG [BaseServerComponent] waitUntilDown: Used 1 delay round(s) to shut down
[Host Controller] 16:34:31,966 INFO  [org.jboss.as.host.controller] (pool-2-thread-16) JBAS010923: Stopping server server-two
16:34:31,972 INFO  [org.jboss.as.process.Server:server-two.status] (ProcessController-threads - 3) JBAS012018: Stopping process 'Server:server-two'
[Host Controller] 16:34:31,972 INFO  [org.jboss.as.host.controller] (pool-2-thread-16) JBAS010923: Stopping server server-one
16:34:31,973 INFO  [org.jboss.as.process.Server:server-one.status] (ProcessController-threads - 3) JBAS012018: Stopping process 'Server:server-one'

However, the operation returns an error:

testDomainServerShutdownAndStartOperations(org.rhq.modules.plugins.jbossas7.itest.domain.DomainServerComponentTest)  Time elapsed: 0.065 sec  <<< FAILURE!
java.lang.AssertionError: Operation [shutdown] with parameters null returned an error: Connection refused, rolled-back=false, rolled-back=false expected:<null> but was:<Connection refused, rolled-back=false, rolled-back=false>
	at org.testng.Assert.fail(Assert.java:89)
	at org.testng.Assert.failNotSame(Assert.java:485)
	at org.testng.Assert.assertNull(Assert.java:421)
	at org.rhq.core.plugin.testutil.AbstractAgentPluginTest.assertOperationSucceeded(AbstractAgentPluginTest.java:337)
	at org.rhq.core.plugin.testutil.AbstractAgentPluginTest.invokeOperationAndAssertSuccess(AbstractAgentPluginTest.java:315)
	at org.rhq.modules.plugins.jbossas7.itest.AbstractServerComponentTest.testShutdownAndStartOperations(AbstractServerComponentTest.java:102)



---

On the standalone side, reload is similar. It appears to work from the logs:

=== Invoking operation [reload] with parameters [null] on Resource[id=2, uuid=11522549-c610-42fd-b079-fdd974361bc2, type={jboss-as-7}JBossAS7 Standalone Server, key=/tmp/jboss-as-7.1.2.Final-SNAPSHOT, name=/tmp/jboss-as-7.1.2.Final-SNAPSHOT, parent=pc, version=7.1.2.Final-SNAPSHOT]...
16:42:21,387 ERROR [ASConnection] Failed to get data: Connection refused
16:42:21,392 ERROR [ASConnection] Failed to get data: Connection refused
16:42:21,393 DEBUG [BaseServerComponent] waitUntilReloaded: Used 1 delay round(s) to reload

but the operation returns an error:

java.lang.AssertionError: Operation [reload] with parameters null returned an error: Connection refused, rolled-back=false, rolled-back=false expected:<null> but was:<Connection refused, rolled-back=false, rolled-back=false>
	at org.testng.Assert.fail(Assert.java:89)
	at org.testng.Assert.failNotSame(Assert.java:485)
	at org.testng.Assert.assertNull(Assert.java:421)
	at org.rhq.core.plugin.testutil.AbstractAgentPluginTest.assertOperationSucceeded(AbstractAgentPluginTest.java:337)
	at org.rhq.core.plugin.testutil.AbstractAgentPluginTest.invokeOperationAndAssertSuccess(AbstractAgentPluginTest.java:315)
	at org.rhq.modules.plugins.jbossas7.itest.standalone.StandaloneServerComponentTest.testReloadOperation(StandaloneServerComponentTest.java:79)


And for shutdown and restart on the standalone side I get:

java.lang.AssertionError: expected:<UP> but was:<DOWN>
	at org.testng.Assert.fail(Assert.java:89)
	at org.testng.Assert.failNotEquals(Assert.java:489)
	at org.testng.Assert.assertEquals(Assert.java:118)
	at org.testng.Assert.assertEquals(Assert.java:160)
	at org.rhq.modules.plugins.jbossas7.itest.standalone.StandaloneServerComponentTest.testRestartOperation(StandaloneServerComponentTest.java:92)


and:

java.lang.AssertionError: expected:<UP> but was:<DOWN>
	at org.testng.Assert.fail(Assert.java:89)
	at org.testng.Assert.failNotEquals(Assert.java:489)
	at org.testng.Assert.assertEquals(Assert.java:118)
	at org.testng.Assert.assertEquals(Assert.java:160)
	at org.rhq.modules.plugins.jbossas7.itest.AbstractServerComponentTest.testShutdownAndStartOperations(AbstractServerComponentTest.java:101)
	at org.rhq.modules.plugins.jbossas7.itest.standalone.StandaloneServerComponentTest.testStandaloneServerShutdownAndStartOperations(StandaloneServerComponentTest.java:85)


So these may actually be bugs in getAvailability() and not restart and shutdown, because the first thing both of these tests do is check that the servers are UP before trying to restart/shutdown them.
Comment 4 Heiko W. Rupp 2012-03-26 11:15:17 EDT
*** Bug 803783 has been marked as a duplicate of this bug. ***
Comment 5 Charles Crouch 2012-03-26 15:04:37 EDT
(1:58:53 PM) ips: the only remaining failures are the start/stop/restart operations
(1:58:59 PM) ips: those tests are currently disabled
(1:59:12 PM) ips: because enabling them would cause all the other tests to fail  
...
(2:02:07 PM) ccrouch: so when this BZ is fixed, we can re-enabled the tests?
(2:02:51 PM) ips: yep
Comment 6 Charles Crouch 2012-04-02 10:36:53 EDT
*** Bug 807942 has been marked as a duplicate of this bug. ***
Comment 7 Charles Crouch 2012-04-02 10:37:39 EDT
Example of where the start op is not working correctly: Bug #807942
Comment 8 Heiko W. Rupp 2012-04-11 07:59:26 EDT
This has been implemented and the hudson tests that test this also work. See also 
Bug 811188  for a (not so realistic) corner case.
Comment 10 Heiko W. Rupp 2013-09-01 06:04:39 EDT
Bulk closing of items that are on_qa and in old RHQ releases, which are out for a long time and where the issue has not been re-opened since.