Bug 1260632

Summary: Storage node resource is not available
Product: [Other] RHQ Project Reporter: Filip Brychta <fbrychta>
Component: Storage NodeAssignee: John Sanda <jsanda>
Status: CLOSED ERRATA QA Contact: Filip Brychta <fbrychta>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: jshepherd
Target Milestone: ---   
Target Release: RHQ 4.14   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-04 22:20:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1208181    

Description Filip Brychta 2015-09-07 11:53:10 UTC
Description of problem:
RHQ Storage Node is not available after installation even though cassandra process is running.
agent.log contains:
2015-09-07 07:33:43,664 WARN  [ResourceContainer.invoker.daemon-1] (org.rhq.plugins.jmx.JMXServerComponent)- Failed to connect to ResourceType[id=0, name=RHQ Storage Node, plugin=RHQStorage, category=Server][RHQ Storage Node(fbr-rhq-test3.bc.jonqe.lab.eng.bos.redhat.com)]: Could not connect [service:jmx:rmi:///jndi/rmi://fbr-rhq-test3.bc.jonqe.lab.eng.bos.redhat.com:7299/jmxrmi] org.mc4j.ems.connection.EmsConnectException: IOException: Check service availability

Version-Release number of selected component (if applicable):
4.14.0-SNAPSHOT

How reproducible:
Always

Steps to Reproduce:
1. install rhq
2. navigate to RHQ Storage Node resource


Actual results:
Resource is down no child resources are discovered

Expected results:
The resource is up

Additional info:
I'm not sure if this is related but I noticed following exception in server.log from completely different setup (it's not visible on clean installation and I don't have repro steps):
12:25:07,026 ERROR [org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerBean] (EJB default - 1) Deployment has been aborted due to failed operation [Announce] on Resource[id=10005, uuid=f6123e50-67db-4945-8048-65ef279d688f, type={RHQStorage}RHQ Storage Node, key=RHQ Storage Node(auto-rhq.bc.jonqe.lab.eng.bos.redhat.com), name=RHQ Storage Node(auto-rhq.bc.jonqe.lab.eng.bos.redhat.com), parent=auto-rhq.bc.jonqe.lab.eng.bos.redhat.com, version=4.14.0-SNAPSHOT]: java.lang.NullPointerException
        at org.rhq.plugins.cassandra.util.KeyspaceService.loadBean(KeyspaceService.java:150)
        at org.rhq.plugins.cassandra.util.KeyspaceService.takeSnapshot(KeyspaceService.java:109)
        at org.rhq.plugins.storage.StorageNodeComponent.createSnapshots(StorageNodeComponent.java:380)
        at org.rhq.plugins.storage.StorageNodeComponent.announce(StorageNodeComponent.java:326)
        at org.rhq.plugins.storage.StorageNodeComponent.invokeOperation(StorageNodeComponent.java:143)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:759)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

Comment 1 Filip Brychta 2015-09-07 11:53:50 UTC
Java version: openJDK 1.7.0_65

Comment 2 John Sanda 2015-09-08 20:41:35 UTC
The Storage Node discovery component needs to make sure that the connectorAddress property uses 127.0.0.1. I have made the changes and pushed to master.

commit hash: bd316143

Comment 3 John Sanda 2015-09-09 02:37:20 UTC
Filip, can you retest this as soon as possible? This blocks bug 1208181. I do not want to merge the changes into the release branch until you have verified in master. Thanks.

Comment 4 Jason Shepherd 2015-11-04 22:20:47 UTC
Fixed as part of RHSA-2015:1947