Bug 536119 (RHQ-503)

Summary: runtime discovery scans executed as post-processing hooks for plugin configuration updates sometimes cause agent-side deadlocks and (eventually) timeouts
Product: [Other] RHQ Project Reporter: Joseph Marques <jmarques>
Component: No ComponentAssignee: Joseph Marques <jmarques>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 1.0Keywords: SubTask
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: All   
URL: http://jira.rhq-project.org/browse/RHQ-503
Whiteboard:
Fixed In Version: 1.0.1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 536058    

Description Joseph Marques 2008-05-19 10:40:00 UTC
steps to reproduce:

* start up 10 secured jboss-as instances (pincipal/credentials should be the same for all of them)
* force an agent discovery if necessary
* import all 10 instances (you'll notice they are red)
* add all 10 to a compatible group
* go to the inventory tab of this group, click edit under "connection properties"
** add the necessary principal/credentials for your servers, click OK

you'll notice that the first or second updates complete in 30-45 seconds, but after that the rest of them seem to take about 10 minutes each.  even though they will look successful, this is somewhat of a false positive.  the update itself completely successfully, but the post-processing hook to execute a runtime discovery timed out.  so, you'll see most (if not all) of the remaining updates take 10 minutes.

these updates are very fast, and should be able to complete within a few minutes.

Comment 1 Joseph Marques 2008-05-19 10:48:59 UTC
rev848: 1.0.0-ga-cp branch
rev849: trunk

remove agent-side deadlock by executing a deferred runtime discovery request after a plugin configuration update; 
fix the method that schedules group plugin configuration updates to prevent hibernate exceptions from happening due to clear/flush methods; 
add a delay in the aggregate plugin configuration update job so that it doesn't try to execute the job before the aggregate update is committed; 

--

following the testing procedures listed in the case, you'll now see that the plugin configurations can be updated against the jbas group of size 10 in approx 3-4 seconds, instead of minutes/hours.

Comment 2 Joseph Marques 2008-06-16 18:58:45 UTC
this was very well tested already since this went out as a patch for a customer several weeks ago.  tested agent on win with 10 jboss, and tested agent on linux with 10 jboss.  server always ran on windows.  confirmed that timings are dramatically reduced over what they used to be.

Comment 3 Red Hat Bugzilla 2009-11-10 21:10:40 UTC
This bug was previously known as http://jira.rhq-project.org/browse/RHQ-503