Description of problem: If a user invoked the agent prompt command: update --update from the RHQ Agent resource's _Execute Prompt Command_ operation, the command is delayed by 299 seconds: 2014-07-29 17:35:06,952 INFO [ResourceContainer.invoker.nonDaemon-2] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.prompt-command-invoked}Prompt command invoked: [update, --update] 2014-07-29 17:35:06,952 INFO [RHQ Agent Update Thread] (org.rhq.enterprise.agent.AgentUpdateThread)- {AgentUpdateThread.started}The agent update thread has started - will begin the agent auto-update now! 2014-07-29 17:35:06,953 INFO [RHQ Agent Update Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.shutting-down}Agent is being shut down... 2014-07-29 17:35:06,953 INFO [RHQ Primary Server Switchover Thread] (org.rhq.enterprise.agent.AgentMain)- {PrimaryServerSwitchoverThread.stopped}The primary server switchover thread has stopped. 2014-07-29 17:35:06,953 INFO [RHQ Agent Update Thread] (org.rhq.core.pc.PluginContainer)- Plugin container is being shutdown... 2014-07-29 17:35:06,953 INFO [RHQ Agent Update Thread] (org.rhq.core.pc.PluginContainer)- Plugin container shutdown will wait up to 300 seconds for shut down background threads to terminate. 2014-07-29 17:35:06,955 INFO [RHQ Agent Update Thread] (rhq.core.pc.content.ContentManager)- Shutting down Content Manager... 2014-07-29 17:35:06,957 ERROR [OperationManager.invoker-1] (rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler)- Thread [OperationManager.invoker-1] was interrupted. 2014-07-29 17:35:06,957 WARN [OperationManager.invoker-1] (rhq.core.pc.inventory.ResourceContainer$ComponentInvocation)- Invocation has been marked interrupted for method [OperationFacet.invokeOperation] on resource [Resource[id=10002, uuid=dfa22364-9154-4532-993e-758876a1f191, type={RHQAgent}RHQ Agent, key=Agent 01164018 on jboss.example.com RHQ Agent, name=RHQ Agent, parent=Agent 01164018 on jboss.example.com, version=4.9.0.JON320GA]] 2014-07-29 17:40:06,954 WARN [RHQ Agent Update Thread] (org.rhq.core.pc.PluginContainer)- Timed out after [299] seconds while waiting for all threads in pool [OperationManager.invoker] to terminate - [1] threads in the pool are still active. ... In the above log excerpt it is clear to see what the issue is. The OperationManager thread is still waiting for the operation to finish. Therefore, it won't shutdown when the agent has asked it to so that the update could proceed. Instead, the agent update thread waits for 299 seconds and then interrupts the thread. Thus, allowing the agent update to proceed. Version-Release number of selected component (if applicable): 3.2.2 How reproducible: Always Steps to Reproduce: 1. Install and start JBoss ON 3.2 system. 2. Install remote agent. 3. Import remote RHQ Agent resource into inventory. 4. Apply 3.2.2 to RHQ_SERVER_HOME. 5. Start JBoss ON server. 6. From UI, invoke remote RHQ Agent's _Execute Prompt Command_ operation with the following _Command_ argument: update --update Actual results: Remote agent does not get updated until over 5 minutes later. Expected results: Remote agent should be updated right away. Additional info: Perhaps the easiest solution for this is to add a new RHQ Agent operation that exposes the same/similar agent prompt command as a standalone operation that can return fast? This allowing the operation manager to shutdown right away instead of waiting?
There are some other (non-prompt-command) ops that workaround this problem. A new op may help get around this, assigning to Mazz, he knows about this stuff.
master commit: commit a894f4becfcaf958d6f92091571fe623bc202fa6 Author: John Mazzitelli <mazz> Date: Mon Sep 8 16:06:48 2014 -0400 BZ 1124619 - provide more responsiveness when you want to ask an agent to update itself from the GUI this needs to be cherry picked
what this now does is provide an actual operation named "Update Agent" - you execute this operation to ask the agent to update itself. You no longer have to go through the "update" prompt command via "Execute Prompt Command" to do this.
stefan cherry picked to 3.3. branch: abb016f51c23896f450eeca7acaa78a435f0a5da
Moving to ON_QA as available for test with the following brew build: https://brewweb.devel.redhat.com//buildinfo?buildID=385149
Verified on version : 3.3.0.ER03 build Number :4aefe39:44e33a4 The agent resource displays 'Update Agent' operation in the drop-down. The operation is executed immediately on the remote agent with success status.