Bug 1214994 - Disperse volume: Rebalance failed when plain disperse volume is converted to distributed disperse volume
Summary: Disperse volume: Rebalance failed when plain disperse volume is converted to ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: disperse
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Pranith Kumar K
QA Contact:
URL:
Whiteboard:
Depends On: 1188145 1211962 1224112 1224181 1225283 1225284
Blocks: qe_tracker_everglades 1224188
TreeView+ depends on / blocked
 
Reported: 2015-04-24 05:06 UTC by Bhaskarakiran
Modified: 2018-10-08 09:53 UTC (History)
3 users (show)

Fixed In Version: glusterfs-3.12.x
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1224188 (view as bug list)
Environment:
Last Closed: 2018-10-08 09:53:32 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Bhaskarakiran 2015-04-24 05:06:33 UTC
Description of problem:
======================

Rebalance failed when a plain disperse 1x(8+4) volume is converted to distributed disperse volume 2x(8+4). 

[root@vertigo ~]# gluster v rebalance testvol status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost              224        11.9MB         29485             0         14532          in progress           48509.00
                                   ninja                0        0Bytes         49629             0             0          in progress           48509.00
                            transformers                0        0Bytes         60960             0             0          in progress           48511.00
                            interstellar                0        0Bytes             0             0             0               failed            1759.00
volume rebalance: testvol: success: 
[root@vertigo ~]# 

Version-Release number of selected component (if applicable):
=============================================================

[root@vertigo ~]# gluster --version
glusterfs 3.7dev built on Apr 18 2015 01:08:09
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@vertigo ~]# 

How reproducible:
=================
100%

Steps to Reproduce:
1. Create a disperse volume 1x(8+4)
2. Create data of around 100GB.
3. Add bricks to convert the disperse volume to distributed disperse volume
4. Trigger rebalance on the volume

Actual results:
==============
Failed

Expected results:


Additional info:
================
sosreports will be copied to rhsqe-repo/sosreports/<budid> folder.

Comment 1 Bhaskarakiran 2015-04-24 05:08:26 UTC
Rebalance is going on from 12 hrs and only 12MB of data is moved.

Comment 2 Bhaskarakiran 2015-04-24 05:09:34 UTC
Volume status and info :
========================

[root@ninja ~]# gluster v status
Status of volume: testvol
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick vertigo:/rhs/brick1/b1                49156     0          Y       21054
Brick ninja:/rhs/brick1/b2                  49152     0          Y       11259
Brick transformers:/rhs/brick1/b3           49152     0          Y       2432 
Brick interstellar:/rhs/brick1/b4           49152     0          Y       2603 
Brick vertigo:/rhs/brick2/b5                49157     0          Y       21071
Brick ninja:/rhs/brick2/b6                  49153     0          Y       11276
Brick transformers:/rhs/brick2/b7           49153     0          Y       2473 
Brick interstellar:/rhs/brick2/b8           49153     0          Y       2620 
Brick vertigo:/rhs/brick3/b9                49158     0          Y       21088
Brick ninja:/rhs/brick3/b10                 49154     0          Y       11293
Brick transformers:/rhs/brick3/b11          49154     0          Y       2490 
Brick interstellar:/rhs/brick3/b12          49154     0          Y       2637 
Brick vertigo:/rhs/brick4/b13               49160     0          Y       21928
Brick ninja:/rhs/brick4/b14                 49156     0          Y       12130
Brick transformers:/rhs/brick4/b15          49156     0          Y       3264 
Brick interstellar:/rhs/brick4/b16          49156     0          Y       3416 
Brick vertigo:/rhs/brick1/b17               49161     0          Y       21947
Brick ninja:/rhs/brick1/b18                 49157     0          Y       12149
Brick transformers:/rhs/brick5/b19          49157     0          Y       3283 
Brick interstellar:/rhs/brick5/b20          49157     0          Y       3435 
Brick vertigo:/rhs/brick2/b21               49162     0          Y       21966
Brick ninja:/rhs/brick2/b22                 49158     0          Y       12168
Brick transformers:/rhs/brick6/b23          49158     0          Y       3302 
Brick interstellar:/rhs/brick6/b24          49158     0          Y       3454 
Snapshot Daemon on localhost                49155     0          Y       11363
NFS Server on localhost                     2049      0          Y       12188
Quota Daemon on localhost                   N/A       N/A        Y       12207
Snapshot Daemon on 10.70.34.56              49159     0          Y       21167
NFS Server on 10.70.34.56                   2049      0          Y       21987
Quota Daemon on 10.70.34.56                 N/A       N/A        Y       22004
Snapshot Daemon on interstellar             49155     0          Y       2707 
NFS Server on interstellar                  2049      0          Y       3477 
Quota Daemon on interstellar                N/A       N/A        Y       3495 
Snapshot Daemon on transformers             49155     0          Y       2562 
NFS Server on transformers                  2049      0          Y       3325 
Quota Daemon on transformers                N/A       N/A        Y       3344 
 
Task Status of Volume testvol
------------------------------------------------------------------------------
Task                 : Rebalance           
ID                   : 5fe22604-3b35-4557-8bd6-8022d7c18905
Status               : in progress         
 
[root@ninja ~]# gluster v i
 
Volume Name: testvol
Type: Distributed-Disperse
Volume ID: 811f6c7d-6b70-4795-8585-d1121b32a673
Status: Started
Number of Bricks: 2 x (8 + 4) = 24
Transport-type: tcp
Bricks:
Brick1: vertigo:/rhs/brick1/b1
Brick2: ninja:/rhs/brick1/b2
Brick3: transformers:/rhs/brick1/b3
Brick4: interstellar:/rhs/brick1/b4
Brick5: vertigo:/rhs/brick2/b5
Brick6: ninja:/rhs/brick2/b6
Brick7: transformers:/rhs/brick2/b7
Brick8: interstellar:/rhs/brick2/b8
Brick9: vertigo:/rhs/brick3/b9
Brick10: ninja:/rhs/brick3/b10
Brick11: transformers:/rhs/brick3/b11
Brick12: interstellar:/rhs/brick3/b12
Brick13: vertigo:/rhs/brick4/b13
Brick14: ninja:/rhs/brick4/b14
Brick15: transformers:/rhs/brick4/b15
Brick16: interstellar:/rhs/brick4/b16
Brick17: vertigo:/rhs/brick1/b17
Brick18: ninja:/rhs/brick1/b18
Brick19: transformers:/rhs/brick5/b19
Brick20: interstellar:/rhs/brick5/b20
Brick21: vertigo:/rhs/brick2/b21
Brick22: ninja:/rhs/brick2/b22
Brick23: transformers:/rhs/brick6/b23
Brick24: interstellar:/rhs/brick6/b24
Options Reconfigured:
client.event-threads: 4
server.event-threads: 4
features.quota: on
features.uss: on
[root@ninja ~]#

Comment 6 Nithya Balachandran 2015-05-12 05:25:43 UTC
Bhaskar confirmed that the failure was because of a crash which is being tracked in BZ#1215117. Hence marking this bug a duplicate of 1215117.

*** This bug has been marked as a duplicate of bug 1215117 ***

Comment 9 Bhaskarakiran 2015-06-12 05:22:57 UTC
Verified this on 3.7.1-1 build and rebalance is successful. Moving this to verified.

Comment 11 Amar Tumballi 2018-10-08 09:53:32 UTC
as per comment #9, this is fixed.


Note You need to log in before you can comment on or make changes to this bug.