Bug 1265279

Summary:	Data Tiering:Quota deamon is restarted when tier is detached (tier volume to regular vol conversion)
Product:	[Community] GlusterFS	Reporter:	Nag Pavan Chilakam <nchilaka>
Component:	tiering	Assignee:	Vijaikumar Mallikarjuna <vmallika>
Status:	CLOSED NOTABUG	QA Contact:	bugs <bugs>
Severity:	medium	Docs Contact:
Priority:	low
Version:	3.7.4	CC:	bugs, nchilaka, smohan, vagarwal
Target Milestone:	---	Keywords:	Reopened
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-09-28 11:35:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Nag Pavan Chilakam 2015-09-22 14:26:48 UTC

Description of problem:
=========================
I created a tier volume and then enabled quotas. I then noted down the quota deamon process IDs. On a detach tier commit, the quota deamons were killed and restarted as below:
[root@zod glusterfs]# gluster v status iran
Status of volume: iran
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick yarrow:/dummy/brick107/iran_hot       49297     0          Y       11353
Brick zod:/dummy/brick107/iran_hot          49325     0          Y       30785
Brick yarrow:/dummy/brick106/iran_hot       49296     0          Y       11373
Brick zod:/dummy/brick106/iran_hot          49324     0          Y       30803
Cold Bricks:
Brick zod:/dummy/brick101/iran              49321     0          Y       30821
Brick yarrow:/dummy/brick101/iran           49293     0          Y       11392
Brick zod:/dummy/brick102/iran              49322     0          Y       30839
Brick yarrow:/dummy/brick102/iran           49294     0          Y       11413
Brick zod:/dummy/brick103/iran              49323     0          Y       30857
Brick yarrow:/dummy/brick103/iran           49295     0          Y       11431
NFS Server on localhost                     2049      0          Y       30884
Quota Daemon on localhost                   N/A       N/A        Y       30893
NFS Server on yarrow                        2049      0          Y       11541
Quota Daemon on yarrow                      N/A       N/A        Y       11552
 
Task Status of Volume iran
------------------------------------------------------------------------------
Task                 : Remove brick        
ID                   : e4b0cd33-5e43-49d1-b5f6-5d558e1a9cf7
Removed bricks:     
zod:/dummy/brick106/iran_hot
yarrow:/dummy/brick106/iran_hot
zod:/dummy/brick107/iran_hot
yarrow:/dummy/brick107/iran_hot
Status               : completed           
 
[root@zod glusterfs]# gluster v detach-tier iran
Usage: volume detach-tier <VOLNAME>  <start|stop|status|commit|[force]>
Tier command failed
[root@zod glusterfs]# gluster v detach-tier iran commit
volume detach-tier commit: success
Check the detached bricks to ensure all files are migrated.
If files with data are found on the brick path, copy them via a gluster mount point before re-purposing the removed brick. 
[root@zod glusterfs]# gluster v detach-tier iran status
volume detach-tier status: failed: volume iran is not a tier volume.
Tier command failed
[root@zod glusterfs]# gluster v status iran
Status of volume: iran
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick zod:/dummy/brick101/iran              49321     0          Y       30821
Brick yarrow:/dummy/brick101/iran           49293     0          Y       11392
Brick zod:/dummy/brick102/iran              49322     0          Y       30839
Brick yarrow:/dummy/brick102/iran           49294     0          Y       11413
Brick zod:/dummy/brick103/iran              49323     0          Y       30857
Brick yarrow:/dummy/brick103/iran           49295     0          Y       11431
NFS Server on localhost                     2049      0          Y       31140
Self-heal Daemon on localhost               N/A       N/A        Y       31148
Quota Daemon on localhost                   N/A       N/A        Y       31156
NFS Server on yarrow                        2049      0          Y       11709
Self-heal Daemon on yarrow                  N/A       N/A        Y       11717
Quota Daemon on yarrow                      N/A       N/A        Y       11725
 
Task Status of Volume iran
------------------------------------------------------------------------------
There are no active volume tasks






Version-Release number of selected component (if applicable):
==========================================================
[root@zod glusterfs]# rpm -qa|grep gluster
glusterfs-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-fuse-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-debuginfo-3.7.4-0.33.git1d02d4b.el7.centos.x86_64
glusterfs-api-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-client-xlators-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-server-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-cli-3.7.4-0.43.gitf139283.el7.centos.x86_64
gglusterfs-libs-3.7.4-0.43.gitf139283.el7.centos.x86_64
l[root@zod glusterfs]# gluster --version
glusterfs 3.7.4 built on Sep 19 2015 01:30:43
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.



Steps to Reproduce:
===================
1.create a tier volume
2. on tier vol enable quotas and not down PIDs of quota demaons
3. detach tier start and commit detach tier
4. Once tier is detached completely, quota deamon can be seen as restarted

Comment 1 Vijaikumar Mallikarjuna 2015-09-28 07:02:46 UTC

quota daemon is expected to restart if there are any changes in the client-volume file.

Could you please check if there is a change in quotad volume file '/var/lib/glusterd/quotad/quotad.vol after tier is detached?

Comment 2 Vivek Agarwal 2015-09-28 10:29:45 UTC

Based on comment

Comment 3 Nag Pavan Chilakam 2015-09-28 11:32:51 UTC

yes there are changes  as below:
[root@zod quotad]# ================================================
bash: ================================================: command not found...
[root@zod quotad]# diff quotad.vol quotad.vol.old 
309a310,349
> volume togo-client-9
>     type protocol/client
>     option password a801ab5b-ca7b-48bc-bc88-fb3dcef55c18
>     option username ae487181-5c4c-4081-811e-377954b2ddb0
>     option transport-type tcp
>     option remote-subvolume /rhs/brick7/togo_hot
>     option remote-host yarrow
>     option ping-timeout 42
> end-volume
> 
> volume togo-client-8
>     type protocol/client
>     option password a801ab5b-ca7b-48bc-bc88-fb3dcef55c18
>     option username ae487181-5c4c-4081-811e-377954b2ddb0
>     option transport-type tcp
>     option remote-subvolume /rhs/brick7/togo_hot
>     option remote-host zod
>     option ping-timeout 42
> end-volume
> 
> volume togo-client-7
>     type protocol/client
>     option password a801ab5b-ca7b-48bc-bc88-fb3dcef55c18
>     option username ae487181-5c4c-4081-811e-377954b2ddb0
>     option transport-type tcp
>     option remote-subvolume /rhs/brick6/togo_hot
>     option remote-host yarrow
>     option ping-timeout 42
> end-volume
> 
> volume togo-client-6
>     type protocol/client
>     option password a801ab5b-ca7b-48bc-bc88-fb3dcef55c18
>     option username ae487181-5c4c-4081-811e-377954b2ddb0
>     option transport-type tcp
>     option remote-subvolume /rhs/brick6/togo_hot
>     option remote-host zod
>     option ping-timeout 42
> end-volume
> 
370c410
< volume togo-replicate-0
---
> volume togo-cold-replicate-0
375c415
< volume togo-replicate-1
---
> volume togo-cold-replicate-1
380c420
< volume togo-replicate-2
---
> volume togo-cold-replicate-2
385c425,440
< volume togo
---
> volume togo-cold
>     type cluster/distribute
>     subvolumes togo-cold-replicate-0 togo-cold-replicate-1 togo-cold-replicate-2
> end-volume
> 
> volume togo-hot-replicate-0
>     type cluster/replicate
>     subvolumes togo-client-9 togo-client-8
> end-volume
> 
> volume togo-hot-replicate-1
>     type cluster/replicate
>     subvolumes togo-client-7 togo-client-6
> end-volume
> 
> volume togo-hot-dht
387c442,450
<     subvolumes togo-replicate-0 togo-replicate-1 togo-replicate-2
---
>     subvolumes togo-hot-replicate-0 togo-hot-replicate-1
> end-volume
> 
> volume togo
>     type cluster/tier
>     option tier-demote-frequency 1800
>     option xattr-name trusted.tier-gfid
>     option rule togo-hot-dht
>     subvolumes togo-cold togo-hot-dht
[root@zod quotad]#

Comment 4 Nag Pavan Chilakam 2015-09-28 11:35:46 UTC

this is the current design of quota...so closing this bug