Bug 999508 - storage node stays in bootstrap mode while joining to cluster
storage node stays in bootstrap mode while joining to cluster
Product: RHQ Project
Classification: Other
Component: Installer (Show other bugs)
x86_64 Linux
unspecified Severity high (vote)
: ---
: RHQ 4.9
Assigned To: John Sanda
Mike Foley
Depends On: 1003611
Blocks: 951619
  Show dependency treegraph
Reported: 2013-08-21 08:46 EDT by Armine Hovsepyan
Modified: 2015-09-02 20:01 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-09-24 15:19:30 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
server_log.png (271.68 KB, image/png)
2013-08-21 08:46 EDT, Armine Hovsepyan
no flags Details
storage_node_admin.png (469.84 KB, image/png)
2013-08-21 08:49 EDT, Armine Hovsepyan
no flags Details
multiple-storage-nodes.png (49.10 KB, image/png)
2013-09-09 11:10 EDT, Armine Hovsepyan
no flags Details

  None (edit)
Description Armine Hovsepyan 2013-08-21 08:46:55 EDT
Created attachment 788860 [details]

Description of problem:
storage node stays in bootstrap mode while joining to cluster

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. install 2 storage nodes and connect to each other (via changing seeds in cassandra.yaml and giving both ip's in rhq-storage-auth.conf)
2. install rhq server with storage and agent
3. add first 2 storage node ip's into cassandra.yaml and rhq-storage-auth.conf of 3rd one and start server, storage and agent
4. install storage node with agent and connect to server from step3

Actual results:
storage node installed by step4 is visible in server's gui -> administration -> storage nodes, stays in bootstrap mode.
Exception in server log: ERROR [org.rhq.server.metrics.MetricsServer] (New I/O worker #6) An error occurred while inserting raw data MeasurementDataNumeric[name=Calculated.TotalDiskUsedPercentage, value=0.24, scheduleId=10383, timestamp=1377088844479]: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ONE (2 replica were required but only 1 acknowledged the write)

all 4 steerage nodes  see each other, are in cluster - according to rhq-storage.log on all 4.

Expected results:
all 4 storage nodes see each other, are in cluster, no exceptions in server.log, storage nodes inventoried into server gui have normal mode, finished bootstrapping and adding to maintenance processes, visible in storage nodes administration with nomal mode.

Additional info:
screenshot attached
Comment 1 Armine Hovsepyan 2013-08-21 08:49:59 EDT
Created attachment 788861 [details]
Comment 2 John Sanda 2013-08-26 23:04:52 EDT
Please review the instructions at https://docs.jboss.org/author/display/RHQ/Deploying+Multiple+Storage+Nodes. I think that there was a configuration error during the initial setup of this cluster.
Comment 3 John Sanda 2013-08-31 18:16:55 EDT
Is this a duplicate of bug 1002202?
Comment 4 Armine Hovsepyan 2013-09-02 09:34:03 EDT
the issue differs from 1002202 - and is blocked by #1003611
Comment 5 Armine Hovsepyan 2013-09-09 11:09:38 EDT

2 nodes installed on windows machines, connected to each other -> server installed on windows, storage nodes with agents connected to server  -> storage with agent installed on linux and connected to server on windows -> all 3 having normal status, node after server installation went through installation processes as described in this doc -> https://docs.jboss.org/author/display/RHQ/Deploying+Storage+Nodes
Comment 6 Armine Hovsepyan 2013-09-09 11:10:05 EDT
Created attachment 795660 [details]

Note You need to log in before you can comment on or make changes to this bug.