Bug 962816 - DHT - rebalance - 'gluster volume rebalance <volname> status' says 'not started' for all node but 'Rebalanced-files' and 'run time in secs' shows count and even log says its completed and it has actually migrated data.
Summary: DHT - rebalance - 'gluster volume rebalance <volname> status' says 'not start...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: distribute
Version: 2.1
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: ---
Assignee: Nithya Balachandran
QA Contact: Matt Zywusko
URL:
Whiteboard:
Depends On:
Blocks: 1286085
TreeView+ depends on / blocked
 
Reported: 2013-05-14 13:38 UTC by Rachana Patel
Modified: 2015-11-27 10:42 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1286085 (view as bug list)
Environment:
Last Closed: 2015-11-27 10:40:54 UTC
Embargoed:


Attachments (Terms of Use)

Description Rachana Patel 2013-05-14 13:38:02 UTC
Description of problem:
DHT - rebalance - 'gluster volume rebalance <volname> status' says 'not started' for all node but 'Rebalanced-files' and 'run time in secs' shows count and even log says its completed and it has actually migrated data.

Version-Release number of selected component (if applicable):
3.4.0.7rhs-1.el6rhs.x86_64

How reproducible:


Steps to Reproduce:
[root@fred glusterd]# gluster volume status t1

Status of volume: t1
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick fan.lab.eng.blr.redhat.com:/rhs/brick1/t1		49155	Y	23093
Brick mia.lab.eng.blr.redhat.com:/rhs/brick1/t1		49155	Y	23630
Brick fred.lab.eng.blr.redhat.com:/rhs/brick1/t1	49153	Y	15263
NFS Server on localhost					2049	Y	15273
NFS Server on fdcb0533-eeb3-4054-8265-26558e92e65a	2049	Y	23104
NFS Server on d665808d-a42a-4eac-bf05-ca53c595486d	2049	Y	23641
 
There are no active volume tasks
[root@fred ~]# gluster volume add-brick t1  fred.lab.eng.blr.redhat.com:/rhs/brick1/t2
volume add-brick: success

[root@fred glusterd]# gluster volume rebalance t1 start force
volume rebalance: t1: failed: Another transaction is in progress. Please try again after sometime.

[root@fred glusterd]# gluster volume rebalance t1 start force
volume rebalance: t1: success: Starting rebalance on volume t1 has been successful.
ID: d87dac2b-5f8f-48db-ada4-c0346a86bc7c
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122             0    not started             1.00
volume rebalance: t1: success: 
[root@fred glusterd]# gluster volume rebalance t1 status
                                    Node Rebalanced-files          size       scanned      failures         status run time in secs
                               ---------      -----------   -----------   -----------   -----------   ------------   --------------
                               localhost               19        0Bytes            66             0    not started             1.00
              fan.lab.eng.blr.redhat.com               26        0Bytes           121             0    not started             1.00
              mia.lab.eng.blr.redhat.com               21        0Bytes           122            


check the log 
<snip>
[2013-05-14 14:38:51.091007] I [dht-rebalance.c:872:dht_migrate_file] 0-t1-dht: completed migration of /renamefile40 from subvolume t1-
client-2 to t1-client-0
[2013-05-14 14:38:51.095263] I [dht-rebalance.c:663:dht_migrate_file] 0-t1-dht: /renamefile43: attempting to move from t1-client-2 to t
1-client-0
[2013-05-14 14:38:51.118271] I [dht-rebalance.c:872:dht_migrate_file] 0-t1-dht: completed migration of /renamefile43 from subvolume t1-
client-2 to t1-client-0
[2013-05-14 14:38:51.122528] I [dht-rebalance.c:663:dht_migrate_file] 0-t1-dht: /renamefile45: attempting to move from t1-client-2 to t
1-client-1
[2013-05-14 14:38:51.155628] I [dht-rebalance.c:872:dht_migrate_file] 0-t1-dht: completed migration of /renamefile45 from subvolume t1-
client-2 to t1-client-1
[2013-05-14 14:38:51.160429] I [dht-rebalance.c:663:dht_migrate_file] 0-t1-dht: /renamefile49: attempting to move from t1-client-2 to t
1-client-1
[2013-05-14 14:38:51.193842] I [dht-rebalance.c:872:dht_migrate_file] 0-t1-dht: completed migration of /renamefile49 from subvolume t1-
client-2 to t1-client-1
[2013-05-14 14:38:51.207934] I [dht-rebalance.c:1311:gf_defrag_migrate_data] 0-t1-dht: Migration operation on dir / took 0.69 secs
[2013-05-14 14:38:51.219623] I [dht-rebalance.c:1733:gf_defrag_status_get] 0-glusterfs: Rebalance is completed. Time taken is 1.00 secs
[2013-05-14 14:38:51.219655] I [dht-rebalance.c:1736:gf_defrag_status_get] 0-glusterfs: Files migrated: 19, size: 0, lookups: 66, failu
res: 0
[2013-05-14 14:38:51.220247] W [glusterfsd.c:1011:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3296ce890d] (-->/lib64/libpthread.so.0() [0x3297407851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x40528d]))) 0-: received signum (15), shutting down


  
Actual results:


Expected results:


Additional info:


Note You need to log in before you can comment on or make changes to this bug.