Bug 1026108 - Deploying storage node fails if announcement of another node failed
Deploying storage node fails if announcement of another node failed
Product: JBoss Operations Network
Classification: JBoss
Component: Core Server (Show other bugs)
JON 3.2
Unspecified Unspecified
urgent Severity urgent
: ER07
: JON 3.2.0
Assigned To: John Sanda
Mike Foley
Depends On: 1026088
Blocks: 1012435
  Show dependency treegraph
Reported: 2013-11-03 13:55 EST by John Sanda
Modified: 2014-03-21 20:47 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1026088
Last Closed: 2014-01-02 15:33:14 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
anounce_exception_alert (132.76 KB, image/png)
2013-11-22 11:09 EST, Armine Hovsepyan
no flags Details
nodes (144.78 KB, image/png)
2013-11-22 11:09 EST, Armine Hovsepyan
no flags Details

  None (edit)
Description John Sanda 2013-11-03 13:55:51 EST
+++ This bug was initially created as a clone of Bug #1026088 +++

Description of problem:
Start with a single node cluster. Deploy a second node, N2. Suppose the deployment fails during the announce phase which means N2 will have an operation mode of ANNOUNCE. Now try to deploy a third node, N3. Deployment of N3 will fail as well with an exception like,

09:54:52,310 ERROR [org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerBean] (EJB default - 3) Aborting storage node deployment due to unexpected error while announcing cluster nodes.: javax.ejb.EJBException: javax.persistence.NonUniqueResultException: result returns more than one elements
        at org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerLocal$$$view137.handleAnnounce(Unknown Source) [rhq-server.jar:4.10.0-SNAPSHOT]
        at org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerBean.handleOperationUpdateIfNecessary(StorageNodeOperationsHandlerBean.java:304) [rhq-server.jar:4.10.0-SNAPSHOT]
Caused by: javax.persistence.NonUniqueResultException: result returns more than one elements
        at org.hibernate.ejb.QueryImpl.getSingleResult(QueryImpl.java:293) [hibernate-entitymanager-4.2.0.CR1.jar:4.2.0.CR1]
        at org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerBean.findStorageNodeByMode(StorageNodeOperationsHandlerBean.java:769) [rhq-server.jar:4.10.0-SNAPSHOT]
        at org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerBean.handleAnnounce(StorageNodeOperationsHandlerBean.java:391) [rhq-server.jar:4.10.0-SNAPSHOT]

When the announce operation finishes on N1 and the agent reports the results to the server, the server fetches N3 using JPA's Query.getSingleResult() and the result set includes both N2 and N3. The filtering in the query is too broad. 

The (un)deployment code was designed and intended to allow deployments in scenarios like this where a previous deployment of another node may have failed and it is easier/faster to deploy a completely different node rather than retrying the failed deployment.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

Actual results:

Expected results:

Additional info:
Comment 1 Heiko W. Rupp 2013-11-08 08:30:15 EST
Fix was pushed to master as master commit hash: 7035640df
Comment 2 John Sanda 2013-11-10 21:57:21 EST
Changes have been pushed to release/jon3.2.x branch.

commit hashes:
Comment 3 Simeon Pinder 2013-11-19 10:47:35 EST
Moving to ON_QA as available for testing with new brew build.
Comment 4 Simeon Pinder 2013-11-22 00:13:08 EST
Mass moving all of these from ER6 to target milestone ER07 since the ER6 build was bad and QE was halted for the same reason.
Comment 5 Armine Hovsepyan 2013-11-22 11:09:24 EST
Created attachment 827877 [details]
Comment 6 Armine Hovsepyan 2013-11-22 11:09:51 EST
Created attachment 827878 [details]
Comment 7 Armine Hovsepyan 2013-11-22 11:20:33 EST
verified in er7 for announce failure (screen-shots attached)

Note You need to log in before you can comment on or make changes to this bug.