Bug 874476 - [RHS-C]: Following auto reboot after Add Server, its takes about 5 min for the server status to reach "UP" state
Summary: [RHS-C]: Following auto reboot after Add Server, its takes about 5 min for th...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: rhsc
Version: 2.0
Hardware: All
OS: Linux
low
urgent
Target Milestone: ---
: ---
Assignee: Sahina Bose
QA Contact: Prasanth
URL:
Whiteboard:
Depends On: 726343 891780 928834
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-11-08 09:48 UTC by Prasanth
Modified: 2013-09-23 22:26 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-23 22:26:05 UTC
Embargoed:


Attachments (Terms of Use)
reboot_time_node1 (18.81 KB, image/png)
2012-11-08 09:48 UTC, Prasanth
no flags Details
reboot_time_node2 (10.67 KB, image/png)
2012-11-08 09:52 UTC, Prasanth
no flags Details
engine logs (74.86 KB, text/x-log)
2012-11-08 09:53 UTC, Prasanth
no flags Details
server logs (12.08 KB, text/x-log)
2012-11-08 09:53 UTC, Prasanth
no flags Details

Description Prasanth 2012-11-08 09:48:25 UTC
Created attachment 640675 [details]
reboot_time_node1

Description of problem:

At present, on adding a New Server, the bootstrapping will take place and the server will be rebooted after the required packages are installed/updated. But though the server is back online within 1-2 minutes, for the UI it takes about 300 sec (5 min) for the Host status to be set to "UP" state. Looks like this is the default wait time set in the script.

----
2012-11-08 04:08:23,291 INFO  [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-48) [4f3edad6] Waiting 300 seconds, for server to finish reboot process.
----

Version-Release number of selected component (if applicable):

rhsc-2.0.techpreview2-0.63.20121019gitffbe992.root.el6ev


How reproducible: Always


Steps to Reproduce:

1. Add a new server 
2. After successful bootstrapping, the server will be rebooted and the Host "Status" will be changed to 'reboot'
3. Now, check the time taken for the Host "Status" to reach the "UP" state. You can confirm the same in the "Events" section as well.

--------
2012-Nov-08, 04:22:44 Detected new Host RHS-node2. Host state was set to Up.
	
2012-Nov-08, 04:17:42 Host RHS-node2 installed
	
2012-Nov-08, 04:17:42 Installing Host RHS-node2. Step: Reboot; Details: Rebooting machine.
--------


Actual results: Currently it waits for 300 seconds (5 minutes) for the server to finish reboot process and change the Host status in UI to UP state even though the server is back online within 1-2 minutes. Why the UI takes 5 min for the status change if the bootstrapping is successfully completed before that??



Expected results:

Why don't we mark the status of the Host as UP as soon the server is back online after reboot and reaches a stable state?


Additional info:

This is what seen in the logs during that time:

----------------------------
2012-11-08 04:08:23,231 INFO  [org.ovirt.engine.core.bll.VdsInstaller] (NioProcessor-236) Installation of 10.70.36.53. Received message: <BSTRAP component='VDS Configuration' status='OK'/>. FYI. (Stage: Running second installation script on Host)
2012-11-08 04:08:23,239 INFO  [org.ovirt.engine.core.bll.VdsInstaller] (NioProcessor-236) Installation of 10.70.36.53. Received message: <BSTRAP component='RHEV_INSTALL' status='OK'/>. Stage completed. (Stage: Running second installation script on Host)
2012-11-08 04:08:23,248 INFO  [org.ovirt.engine.core.bll.VdsInstaller] (NioProcessor-236) Installation of 10.70.36.53. Received message: <BSTRAP component='Reboot' status='OK' message='Rebooting machine' />. FYI. (Stage: Host installation complete)
2012-11-08 04:08:23,268 INFO  [org.ovirt.engine.core.bll.VdsInstaller] (pool-4-thread-49) [4f3edad6] Script ended, result is {1}
2012-11-08 04:08:23,269 INFO  [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-49) [4f3edad6] After Installation pool-4-thread-49
2012-11-08 04:08:23,274 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-49) [4f3edad6] START, SetVdsStatusVDSCommand(HostName = RHS-node1, HostId = a17bfb18-292b-11e2-adf2-0025907c2e9e, status=Reboot, nonOperationalReason=NONE), log id: 3
f1bf517
2012-11-08 04:08:23,291 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-49) [4f3edad6] FINISH, SetVdsStatusVDSCommand, log id: 3f1bf517
2012-11-08 04:08:23,291 INFO  [org.ovirt.engine.core.bll.InstallVdsCommand] (pool-4-thread-48) [4f3edad6] Waiting 300 seconds, for server to finish reboot process.
2012-11-08 04:10:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-82) Autorecovering hosts is disabled, skipping
2012-11-08 04:10:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-82) Autorecovering storage domains is disabled, skipping
2012-11-08 04:13:23,293 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-48) [4f3edad6] START, SetVdsStatusVDSCommand(HostName = RHS-node1, HostId = a17bfb18-292b-11e2-adf2-0025907c2e9e, status=NonResponsive, nonOperationalReason=NONE), log id: 4500e662
2012-11-08 04:13:23,312 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-48) [4f3edad6] FINISH, SetVdsStatusVDSCommand, log id: 4500e662
2012-11-08 04:13:25,969 INFO  [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (QuartzScheduler_Worker-92) [c36dcf3] Running command: InitVdsOnUpCommand internal: true.
----------------------------

engine/server logs and screenshots of "Events" are attached! Let me know if you need any additional information.

Comment 1 Prasanth 2012-11-08 09:52:39 UTC
Created attachment 640678 [details]
reboot_time_node2

Comment 2 Prasanth 2012-11-08 09:53:21 UTC
Created attachment 640685 [details]
engine logs

Comment 3 Prasanth 2012-11-08 09:53:46 UTC
Created attachment 640687 [details]
server logs

Comment 7 Alon Bar-Lev 2012-11-16 13:10:52 UTC
This is dup of bug#726343.

Comment 9 Sahina Bose 2013-04-23 04:58:01 UTC
As per bug#928834, RHSC hosts do not require reboot. So marking this bug ON_QA as scenario is not valid any more.

Comment 10 Prasanth 2013-04-30 09:58:09 UTC
As per the fix introduced in Bug 928834, marking this bug as Verified.

Comment 11 Scott Haines 2013-09-23 22:26:05 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html


Note You need to log in before you can comment on or make changes to this bug.