Bug 784590 - [eap6] Unable to retrieve current configuration after updating Resource Configuration Timed out
[eap6] Unable to retrieve current configuration after updating Resource Confi...
Product: RHQ Project
Classification: Other
Component: Agent, Core Server, Configuration, Plugins (Show other bugs)
Unspecified Linux
high Severity high (vote)
: ---
: ---
Assigned To: Simeon Pinder
Mike Foley
Depends On:
Blocks: as7-plugin
  Show dependency treegraph
Reported: 2012-01-25 08:51 EST by Libor Zoubek
Modified: 2015-11-01 19:42 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2012-02-28 11:07:11 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
lzoubek: needinfo+

Attachments (Terms of Use)

  None (edit)
Description Libor Zoubek 2012-01-25 08:51:17 EST
Description of problem: When verifying https://bugzilla.redhat.com/show_bug.cgi?id=708332 it happened to me, that Update Configuration job failed, because it timed out (do not know why it timed out). What is worse, since that I am not able to retrieve current configuration of affected resource.

Version-Release number of selected component (if applicable):
Version: 4.3.0-SNAPSHOT
Build Number: 8125902


How reproducible: hard

Steps to Reproduce:
1. Invoke a resource configuraion change - Add system property to EAP6  DomainController
2. Let the operation timeout (not sure how to reproduce)
3. Wait for RHQ UI to recognize that configuration change failed
4. Check Configuration change history - last change should be marked as failed with timeout message - correct
5. Try to see 'Current' configuration of EAP6 DomainController
Actual results:

StackTrace of failed operation on agent:

2012-01-25 14:23:36,871 ERROR [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.UpdateResourceConfigurationRunner)- Plugin Error: Exception thrown while updating Resource configuration for JBossAS7 Host Controller Resource with id [10013].
org.rhq.core.pc.inventory.TimeoutException: Call to [org.rhq.modules.plugins.jbossas7.HostControllerComponent.updateResourceConfiguration()] with args [[org.rhq.core.pluginapi.configuration.ConfigurationUpdateReport@1b834b9]] timed out. Invocation thread will be interrupted
        at org.rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler.invokeInNewThreadWithLock(ResourceContainer.java:492)
        at org.rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler.invoke(ResourceContainer.java:461)
        at $Proxy86.updateResourceConfiguration(Unknown Source)
        at org.rhq.core.pc.configuration.LegacyConfigManagement.executeUpdate(LegacyConfigManagement.java:87)
        at org.rhq.core.pc.configuration.UpdateResourceConfigurationRunner.call(UpdateResourceConfigurationRunner.java:121)
        at org.rhq.core.pc.configuration.UpdateResourceConfigurationRunner.run(UpdateResourceConfigurationRunner.java:109)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)

After that, a warning in server log, when attempting to see resource's current configuration

WARN  [org.rhq.enterprise.server.configuration.ConfigurationManagerBean] Could not get live resource configuration for resource [Resource[id=10013, type=JBossAS7 Host Controller, key=DomainController, name=EAP DomainController, parent=dhcp-1-216.brq.redhat.com, version=EAP domain]]; will assume latest resource configuration update is the current resource configuration.

Expected results: Even when resource change 

Additional info:
Comment 1 Heiko W. Rupp 2012-01-30 06:06:33 EST
I can't reproduce this.
Comment 2 Libor Zoubek 2012-01-30 06:53:50 EST
Have you succeeded to make configuration update to timeout? I can't reproduce the timeout either. I'll write automated tests for https://bugzilla.redhat.com/show_bug.cgi?id=708332 soon, so they could help discover whether timeouts in this case are random or do not happen at all.
Comment 3 Heiko W. Rupp 2012-01-30 07:13:09 EST
Well, not time out, but not succeed by stopping the DC
Comment 5 Mike Foley 2012-01-30 11:23:34 EST
12/30/2012 BZ triage meeting mfoley, ccrouch, loleary, asantos
Comment 6 Heiko W. Rupp 2012-02-14 12:07:50 EST
This has been identified by Libor as being a bug in as7 (he was able to hang it with the as7-cli as well).

To prevent hangs of the as7 plugin, a timeout has been added so that a hanging connection is terminated after 10 secs.

Libor wanted to report an as7 issue
Comment 7 Simeon Pinder 2012-02-23 04:40:12 EST
I am also unable to reproduce this. I did modify the plugin to generate resource errors when the AS7 communication is down because of invalid credentials.  The resource error icon shows up in the ui when trying to browse a resource. 

I'm guessing that Libor is seeing this because he's switching in between HC and Standalone modes and perhaps the communication over resources is getting confused. Either way with the new timeout and more information about when plugin operations cannot happen because of plugin problems we may not see this anymore.
Comment 8 Simeon Pinder 2012-02-23 09:44:10 EST
Hi Libor, Can you try to reproduce this again?  Not being able to replicate this is making it difficult to find and fix if it's still there.
Comment 9 Libor Zoubek 2012-02-28 11:07:11 EST
I cannot reproduce it either.

Note You need to log in before you can comment on or make changes to this bug.