Bug 1014961

Summary: on HA setup storage node database is not sharing
Product: [JBoss] JBoss Operations Network Reporter: Jeeva Kandasamy <jkandasa>
Component: High AvailabilityAssignee: Stefan Negrea <snegrea>
Status: CLOSED CURRENTRELEASE QA Contact: Mike Foley <mfoley>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: JON 3.2CC: jkandasa, jsanda, myarboro, snegrea
Target Milestone: ER04   
Target Release: JON 3.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1012435    
Attachments:
Description Flags
metrics data not available
none
log files
none
log files none

Description Jeeva Kandasamy 2013-10-03 08:26:20 UTC
Created attachment 806919 [details]
metrics data not available

Description of problem:
On JON HA setup, storage node database is not sharing.
Two JON nodes on HA setup, Node-A and Node-B, If we see Node-A resources metric  from Node-B server GUI it's not available and vice versa

Version-Release number of selected component (if applicable):
Version: 3.2.0.ER2
Build Number: 9bf6f76:371eac0
GWT Version: 2.5.0
SmartGWT Version: 3.0

How reproducible:
always

Steps to Reproduce:
1. Create HA setup at least with two JON servers.
2. Launch Node-A resource metric from Node-B server GUI

Actual results:
No data

Expected results:
storage data also should be on HA setup

Additional info: screen is attached

Comment 1 Jeeva Kandasamy 2013-10-03 13:14:23 UTC
Hope this can be resolved If we have give single point storage database on HA setup, we might provide clear document to configure storage node database as like postgres/oracle database on HA setup.

Comment 2 John Sanda 2013-10-07 18:00:56 UTC
Hi Jeeva,

Can you provide log files for the second server installer as well as the storage node logs? You can find them at,

<rhq-server-dir>/logs/rhq-installer.log
<rhq-server-dir>/logs/server.log
<rhq-server-dir>/logs/rhq-storage.log

Can you also provide more details about how you installed the second server/storage node? Thanks.

Comment 3 Jeeva Kandasamy 2013-10-08 07:39:00 UTC
Hi John,

As storage node is new to HA setup, I followed doc from RHQ (https://docs.jboss.org/author/display/RHQ/High+Availability). I didn't find any document to share storage node on HA setup. I'm thinking that storage node will be shared based on the data that we have in RDBMS database (postgres/oracle). As we are giving single point RDBMS database for all JON server(s). 

Steps I followed,

- Installed node-A on a machine (RHQ server + storage node + RDBMS database)
- Installed node-B on another machine (RHQ server + storage node), pointed RDBMS database to node-A.

I didn't do any special setup for storage node. I hope problem will be solved if we share storage node across. But I couldn't find any document to share storage node on HA setup.

Comment 4 Jeeva Kandasamy 2013-10-08 07:40:14 UTC
Created attachment 809149 [details]
log files

log files are attached from node-A and node-B

<rhq-server-dir>/logs/rhq-installer.log
<rhq-server-dir>/logs/server.log
<rhq-server-dir>/logs/rhq-storage.log

Comment 5 Jeeva Kandasamy 2013-10-08 07:43:48 UTC
Created attachment 809150 [details]
log files

Comment 6 John Sanda 2013-10-08 20:10:51 UTC
The storage node on each machine is bound to 127.0.0.1, so they will not be able to communicate with each other. If the hostname is not set for the storage installer, then it defaults to InetAddress.getLocalHost() which may or may not return localhost depending on how your hostname is configured. Review the documentation at https://docs.jboss.org/author/display/RHQ/RHQ+Server+Installation for details about installing with the storage node.

I should point out that bug 1015835 impacts the installation in HA deployments. If you retest before bug 1015835 is fixed, I can help you work around it.

Comment 7 Stefan Negrea 2013-10-16 20:46:03 UTC
Update the installer code to always storage cluster information from the database if it was previously persisted.

release/jon3.2.x branch commits:

https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/?h=release/jon3.2.x&id=0facc965c95822a6b704015662ad79c15f60d37e

https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/?h=release/jon3.2.x&id=dc0ed2095a239abeb6a3c956af2fcacf7af1f511

Comment 8 Simeon Pinder 2013-10-24 04:10:19 UTC
Moving to ON_QA for testing in the next build.

Comment 10 Jeeva Kandasamy 2013-10-29 12:16:16 UTC
I'm able to create JON server on HA setup. Storage node data are in sync. Creating second JON server with storage node on HA setup doesn't give any issue. However I'm unable to create second JON server without storage node. To track this I have created another bug # https://bugzilla.redhat.com/show_bug.cgi?id=1024326

Hence I'm moving this bug to verified state.