Bug 1237056

Description of problem:
========================
If a detach-tier has been started and if in between, the detach-tier has been stopped for some reason, then the gluster deamon should restart by itself.
Currently, once the detach-tier has been issued, the tier deamon stops and is not restarting on issue of detach-tier stop

Version-Release number of selected component (if applicable):
=============================================================
[root@tettnang ~]# rpm -qa|grep gluster
glusterfs-geo-replication-3.7.1-6.el7rhgs.x86_64
glusterfs-api-3.7.1-6.el7rhgs.x86_64
glusterfs-3.7.1-6.el7rhgs.x86_64
glusterfs-cli-3.7.1-6.el7rhgs.x86_64
glusterfs-debuginfo-3.7.1-6.el7rhgs.x86_64
glusterfs-client-xlators-3.7.1-6.el7rhgs.x86_64
glusterfs-server-3.7.1-6.el7rhgs.x86_64
glusterfs-fuse-3.7.1-6.el7rhgs.x86_64
glusterfs-libs-3.7.1-6.el7rhgs.x86_64
glusterfs-rdma-3.7.1-6.el7rhgs.x86_64
[root@tettnang ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.1 (Maipo)
[root@tettnang ~]# gluster --version
glusterfs 3.7.1 built on Jun 28 2015 11:01:14
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.



How reproducible:
==================
easily


Steps to Reproduce:
===================
1.create a tier volume and have it mounted
2.you can see that the tier deamon has been started(check in /var/lib/glusterd/vols/<volname>/tier/)
3.also you can view "gluster v rebalance tier status" and "gluster v status" and "gluster v rebalance status" to see the tier rebalance running
4. Now add huge number of files
5. Do a detach-tier start
6. Now when the rebalancer moves the files to cold, do a detach-tier stop
7. It can now be seen that the rebalance would have stopped completely. This is not expected.
The rebalancer should restart once detach-tier has been stopped.

Alternative method:
1-3 as above
4. Add only a few files
5. do a detach-tier start
6. now keep checking volume rebalance tier status and wait for rebalance to complete and the deamon should have stopped
7. Now do a tier detach-tier stop. This should restart the deamon




Actual results:
================
tier deamon doesnt restart when user stops detach-tier

Expected results:
==================
tier deamon should restart when user issues detach-tier stop


Additional Info:
===============
Raising this as a low severity bug as the user has a work-around of issuing gluster v rebalance <volname> tier start anytime


[root@tettnang ~]# gluster v status
Status of volume: vol101
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/rhs/brick7/vol101             49153     0          Y       9602 
Brick tettnang:/rhs/brick7/vol101           49153     0          Y       32767
Cold Bricks:
Brick tettnang:/rhs/brick1/vol101           49152     0          Y       319  
Brick zod:/rhs/brick1/vol101                49152     0          Y       21640
Brick zod:/rhs/brick2/vol101                49153     0          Y       21661
Brick yarrow:/rhs/brick1/vol101             49152     0          Y       9620 
NFS Server on localhost                     N/A       N/A        N       N/A  
NFS Server on yarrow                        N/A       N/A        N       N/A  
NFS Server on zod                           N/A       N/A        N       N/A  
 
Task Status of Volume vol101
------------------------------------------------------------------------------
Task                 : Rebalance           
ID                   : e626a706-3b81-4db9-9842-d82dcc0852c0
Status               : in progress         
 
[root@tettnang ~]# gluster v detach-tier vol101 status
volume detach-tier status: failed: Detach-tier not started.
[root@tettnang ~]# gluster v rebalance vol101 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0          in progress             729.00
                                     zod                0        0Bytes             0             0             0          in progress             729.00
                                  yarrow                0        0Bytes             0             0             0          in progress             728.00
volume rebalance: vol101: success: 
[root@tettnang ~]# gluster v rebalance vol101 tier status
Node                 Promoted files       Demoted files        Status              
---------            ---------            ---------            ---------           
localhost            0                    0                    in progress         
zod                  0                    0                    in progress         
yarrow               0                    0                    in progress         
volume rebalance: vol101: success: 
[root@tettnang ~]# gluster v detach-tier vol101 start
volume detach-tier start: success
ID: baac190e-5b69-4e15-98ef-71328c276fe0
[root@tettnang ~]# gluster v detach-tier vol101
Usage: volume detach-tier <VOLNAME>  <start|stop|status|commit|[force]>
[root@tettnang ~]# gluster v detach-tier vol101 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
                                  yarrow                0        0Bytes             0             0             0            completed               0.00
[root@tettnang ~]# gluster v rebalance vol101 tier status
volume rebalance: vol101: failed: Rebalance not started.
[root@tettnang ~]# gluster v status vol101
Status of volume: vol101
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/rhs/brick7/vol101             49153     0          Y       9602 
Brick tettnang:/rhs/brick7/vol101           49153     0          Y       32767
Cold Bricks:
Brick tettnang:/rhs/brick1/vol101           49152     0          Y       319  
Brick zod:/rhs/brick1/vol101                49152     0          Y       21640
Brick zod:/rhs/brick2/vol101                49153     0          Y       21661
Brick yarrow:/rhs/brick1/vol101             49152     0          Y       9620 
NFS Server on localhost                     N/A       N/A        N       N/A  
NFS Server on zod                           N/A       N/A        N       N/A  
NFS Server on yarrow                        N/A       N/A        N       N/A  
 
Task Status of Volume vol101
------------------------------------------------------------------------------
Task                 : Remove brick        
ID                   : baac190e-5b69-4e15-98ef-71328c276fe0
Removed bricks:     
tettnang:/rhs/brick7/vol101
yarrow:/rhs/brick7/vol101
Status               : completed           
 
[root@tettnang ~]# gluster v detach-tier vol101 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
                                  yarrow                0        0Bytes             0             0             0            completed               0.00
[root@tettnang ~]# gluster v detach-tier vol101 stop
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
                                  yarrow                0        0Bytes             0             0             0            completed               0.00
'detach-tier' process may be in the middle of a file migration.
The process will be fully stopped once the migration of the file is complete.
Please check detach-tier process for completion before doing any further brick related tasks on the volume.
[root@tettnang ~]# gluster v status vol101
Status of volume: vol101
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/rhs/brick7/vol101             49153     0          Y       9602 
Brick tettnang:/rhs/brick7/vol101           49153     0          Y       32767
Cold Bricks:
Brick tettnang:/rhs/brick1/vol101           49152     0          Y       319  
Brick zod:/rhs/brick1/vol101                49152     0          Y       21640
Brick zod:/rhs/brick2/vol101                49153     0          Y       21661
Brick yarrow:/rhs/brick1/vol101             49152     0          Y       9620 
NFS Server on localhost                     N/A       N/A        N       N/A  
NFS Server on zod                           N/A       N/A        N       N/A  
NFS Server on yarrow                        N/A       N/A        N       N/A  
 
Task Status of Volume vol101
------------------------------------------------------------------------------
There are no active volume tasks
 
[root@tettnang ~]# gluster v detach-tier vol101 status
volume detach-tier status: failed: Detach-tier not started.
[root@tettnang ~]#