Bug 1237056 - Data Tiering:tiering deamon should restart if detach-tier has been stopped
Summary: Data Tiering:tiering deamon should restart if detach-tier has been stopped
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: tier
Version: rhgs-3.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Mohammed Rafi KC
QA Contact: Nag Pavan Chilakam
URL:
Whiteboard: tier-attach-detach
Depends On:
Blocks: 1276742
TreeView+ depends on / blocked
 
Reported: 2015-06-30 09:52 UTC by Nag Pavan Chilakam
Modified: 2018-11-08 19:17 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-08 19:17:46 UTC
Embargoed:


Attachments (Terms of Use)

Description Nag Pavan Chilakam 2015-06-30 09:52:07 UTC
Description of problem:
========================
If a detach-tier has been started and if in between, the detach-tier has been stopped for some reason, then the gluster deamon should restart by itself.
Currently, once the detach-tier has been issued, the tier deamon stops and is not restarting on issue of detach-tier stop

Version-Release number of selected component (if applicable):
=============================================================
[root@tettnang ~]# rpm -qa|grep gluster
glusterfs-geo-replication-3.7.1-6.el7rhgs.x86_64
glusterfs-api-3.7.1-6.el7rhgs.x86_64
glusterfs-3.7.1-6.el7rhgs.x86_64
glusterfs-cli-3.7.1-6.el7rhgs.x86_64
glusterfs-debuginfo-3.7.1-6.el7rhgs.x86_64
glusterfs-client-xlators-3.7.1-6.el7rhgs.x86_64
glusterfs-server-3.7.1-6.el7rhgs.x86_64
glusterfs-fuse-3.7.1-6.el7rhgs.x86_64
glusterfs-libs-3.7.1-6.el7rhgs.x86_64
glusterfs-rdma-3.7.1-6.el7rhgs.x86_64
[root@tettnang ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.1 (Maipo)
[root@tettnang ~]# gluster --version
glusterfs 3.7.1 built on Jun 28 2015 11:01:14
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.



How reproducible:
==================
easily


Steps to Reproduce:
===================
1.create a tier volume and have it mounted
2.you can see that the tier deamon has been started(check in /var/lib/glusterd/vols/<volname>/tier/)
3.also you can view "gluster v rebalance tier status" and "gluster v status" and "gluster v rebalance status" to see the tier rebalance running
4. Now add huge number of files
5. Do a detach-tier start
6. Now when the rebalancer moves the files to cold, do a detach-tier stop
7. It can now be seen that the rebalance would have stopped completely. This is not expected.
The rebalancer should restart once detach-tier has been stopped.

Alternative method:
1-3 as above
4. Add only a few files
5. do a detach-tier start
6. now keep checking volume rebalance tier status and wait for rebalance to complete and the deamon should have stopped
7. Now do a tier detach-tier stop. This should restart the deamon




Actual results:
================
tier deamon doesnt restart when user stops detach-tier

Expected results:
==================
tier deamon should restart when user issues detach-tier stop


Additional Info:
===============
Raising this as a low severity bug as the user has a work-around of issuing gluster v rebalance <volname> tier start anytime


[root@tettnang ~]# gluster v status
Status of volume: vol101
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/rhs/brick7/vol101             49153     0          Y       9602 
Brick tettnang:/rhs/brick7/vol101           49153     0          Y       32767
Cold Bricks:
Brick tettnang:/rhs/brick1/vol101           49152     0          Y       319  
Brick zod:/rhs/brick1/vol101                49152     0          Y       21640
Brick zod:/rhs/brick2/vol101                49153     0          Y       21661
Brick yarrow:/rhs/brick1/vol101             49152     0          Y       9620 
NFS Server on localhost                     N/A       N/A        N       N/A  
NFS Server on yarrow                        N/A       N/A        N       N/A  
NFS Server on zod                           N/A       N/A        N       N/A  
 
Task Status of Volume vol101
------------------------------------------------------------------------------
Task                 : Rebalance           
ID                   : e626a706-3b81-4db9-9842-d82dcc0852c0
Status               : in progress         
 
[root@tettnang ~]# gluster v detach-tier vol101 status
volume detach-tier status: failed: Detach-tier not started.
[root@tettnang ~]# gluster v rebalance vol101 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0          in progress             729.00
                                     zod                0        0Bytes             0             0             0          in progress             729.00
                                  yarrow                0        0Bytes             0             0             0          in progress             728.00
volume rebalance: vol101: success: 
[root@tettnang ~]# gluster v rebalance vol101 tier status
Node                 Promoted files       Demoted files        Status              
---------            ---------            ---------            ---------           
localhost            0                    0                    in progress         
zod                  0                    0                    in progress         
yarrow               0                    0                    in progress         
volume rebalance: vol101: success: 
[root@tettnang ~]# gluster v detach-tier vol101 start
volume detach-tier start: success
ID: baac190e-5b69-4e15-98ef-71328c276fe0
[root@tettnang ~]# gluster v detach-tier vol101
Usage: volume detach-tier <VOLNAME>  <start|stop|status|commit|[force]>
[root@tettnang ~]# gluster v detach-tier vol101 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
                                  yarrow                0        0Bytes             0             0             0            completed               0.00
[root@tettnang ~]# gluster v rebalance vol101 tier status
volume rebalance: vol101: failed: Rebalance not started.
[root@tettnang ~]# gluster v status vol101
Status of volume: vol101
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/rhs/brick7/vol101             49153     0          Y       9602 
Brick tettnang:/rhs/brick7/vol101           49153     0          Y       32767
Cold Bricks:
Brick tettnang:/rhs/brick1/vol101           49152     0          Y       319  
Brick zod:/rhs/brick1/vol101                49152     0          Y       21640
Brick zod:/rhs/brick2/vol101                49153     0          Y       21661
Brick yarrow:/rhs/brick1/vol101             49152     0          Y       9620 
NFS Server on localhost                     N/A       N/A        N       N/A  
NFS Server on zod                           N/A       N/A        N       N/A  
NFS Server on yarrow                        N/A       N/A        N       N/A  
 
Task Status of Volume vol101
------------------------------------------------------------------------------
Task                 : Remove brick        
ID                   : baac190e-5b69-4e15-98ef-71328c276fe0
Removed bricks:     
tettnang:/rhs/brick7/vol101
yarrow:/rhs/brick7/vol101
Status               : completed           
 
[root@tettnang ~]# gluster v detach-tier vol101 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
                                  yarrow                0        0Bytes             0             0             0            completed               0.00
[root@tettnang ~]# gluster v detach-tier vol101 stop
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
                                  yarrow                0        0Bytes             0             0             0            completed               0.00
'detach-tier' process may be in the middle of a file migration.
The process will be fully stopped once the migration of the file is complete.
Please check detach-tier process for completion before doing any further brick related tasks on the volume.
[root@tettnang ~]# gluster v status vol101
Status of volume: vol101
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/rhs/brick7/vol101             49153     0          Y       9602 
Brick tettnang:/rhs/brick7/vol101           49153     0          Y       32767
Cold Bricks:
Brick tettnang:/rhs/brick1/vol101           49152     0          Y       319  
Brick zod:/rhs/brick1/vol101                49152     0          Y       21640
Brick zod:/rhs/brick2/vol101                49153     0          Y       21661
Brick yarrow:/rhs/brick1/vol101             49152     0          Y       9620 
NFS Server on localhost                     N/A       N/A        N       N/A  
NFS Server on zod                           N/A       N/A        N       N/A  
NFS Server on yarrow                        N/A       N/A        N       N/A  
 
Task Status of Volume vol101
------------------------------------------------------------------------------
There are no active volume tasks
 
[root@tettnang ~]# gluster v detach-tier vol101 status
volume detach-tier status: failed: Detach-tier not started.
[root@tettnang ~]#

Comment 3 Mohammed Rafi KC 2015-12-01 09:32:10 UTC
upstream patch : http://review.gluster.org/12833

Comment 4 Mohammed Rafi KC 2016-03-23 07:00:52 UTC
This bug is already fixed in master, and glusterfs-3.7.9 (review.gluster.org/#/c/12921/).

Comment 5 Mike McCune 2016-03-28 23:28:22 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 6 Milind Changire 2017-01-18 09:54:37 UTC
Moving to MODIFIED
Patch available downstream as commit 6746376.

Comment 9 hari gowtham 2018-11-08 19:17:46 UTC
As tier is not being actively developed, I'm closing this bug. Feel free to open it if necessary.


Note You need to log in before you can comment on or make changes to this bug.