Bug 1015659

Summary: [RHSC] Even though rebalance is still in progress, the icon on the UI indicates that it has been completed.
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Shruti Sampat <ssampat>
Component: rhscAssignee: Ramesh N <rnachimu>
Status: CLOSED ERRATA QA Contact: Shruti Sampat <ssampat>
Severity: high Docs Contact:
Priority: high    
Version: 2.1CC: dpati, dtsang, knarra, mmahoney, pprakash, rhs-bugs, sdharane
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 2.1.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: cb7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-25 07:50:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1020331, 1027094    
Bug Blocks:    
Attachments:
Description Flags
engine logs
none
vdsm logs - server 1
none
vdsm logs - server 2
none
vdsm logs - server 3
none
vdsm logs - server 4
none
Attaching screen shots
none
Attaching screen shots
none
Attaching screen shots none

Description Shruti Sampat 2013-10-04 18:28:46 UTC 
Description of problem:
------------------------
After starting rebalance on a volume from the UI, the icon in the activities column for that volume, reflects that rebalance is running. After a while the icon shows a tick mark, which indicates that the rebalance is now completed. But according to what is shown by the glusterfs CLI output of rebalance status command, rebalance was still in progress on some of the nodes.

Version-Release number of selected component (if applicable):
Red Hat Storage Console Version: 2.1.1-0.0.2.master.el6ev 

How reproducible:
Frequently

Steps to Reproduce:
1. Create a volume, start it and mount it on a client.
2. Create some data at the mount point and start rebalance on the volume from the Console.
3. Check the status of rebalance at the gluster CLI, and check the icon in the activities icon on the Console, for that volume.

Actual results:
After a while, the activities icon for that volume changes to indicate that rebalance has been completed. But the glusterfs CLI status says that rebalance is still running on some of the nodes.

Expected results:
As long as the rebalance process is running on any of the nodes, rebalance should be shown as 'in progress' on the Console. 

Additional info:
Find engine and vdsm logs from all the nodes in the cluster.
Comment 1 Shruti Sampat 2013-10-04 18:47:17 UTC 
Created attachment 807899 [details]
engine logs
Comment 2 Shruti Sampat 2013-10-04 18:48:43 UTC 
Created attachment 807900 [details]
vdsm logs - server 1
Comment 3 Shruti Sampat 2013-10-04 18:50:01 UTC 
Created attachment 807901 [details]
vdsm logs - server 2
Comment 4 Shruti Sampat 2013-10-04 18:50:56 UTC 
Created attachment 807902 [details]
vdsm logs - server 3
Comment 5 Shruti Sampat 2013-10-04 18:51:29 UTC 
Created attachment 807903 [details]
vdsm logs - server 4
Comment 7 RamaKasturi 2013-10-15 07:14:15 UTC 
The above happens for distribute replicate volume as well, when rebalance is inprogress for two nodes.
Comment 8 RamaKasturi 2013-10-17 12:50:36 UTC 
Created attachment 813296 [details]
Attaching screen shots
Comment 9 RamaKasturi 2013-10-17 12:51:20 UTC 
Created attachment 813297 [details]
Attaching screen shots
Comment 10 RamaKasturi 2013-10-17 12:53:20 UTC 
Created attachment 813298 [details]
Attaching screen shots
Comment 11 Ramesh N 2013-10-21 11:38:04 UTC 
This issue is because of gluster CLI issue reported in bug https://bugzilla.redhat.com/show_bug.cgi?id=1020331. Different nodes in the gluster gives different status for the same task.
Comment 12 Shruti Sampat 2013-11-13 14:10:10 UTC 
Verified as fixed in Red Hat Storage Console Version: 2.1.2-0.23.master.el6_5.
Comment 13 Ramesh N 2014-01-03 11:53:49 UTC 
Doc Text is not required  as its a new feature and we already have an RFE
Comment 15 errata-xmlrpc 2014-02-25 07:50:27 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0208.html