Bug 1232625 - Data Tiering: Files not getting promoted once demoted
Summary: Data Tiering: Files not getting promoted once demoted
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: tier
Version: rhgs-3.1
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: RHGS 3.1.0
Assignee: Joseph Elwin Fernandes
QA Contact: Nag Pavan Chilakam
URL:
Whiteboard:
Depends On:
Blocks: 1202842 1235269 1236288
TreeView+ depends on / blocked
 
Reported: 2015-06-17 07:54 UTC by Nag Pavan Chilakam
Modified: 2016-09-17 15:40 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.7.1-6
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1235269 (view as bug list)
Environment:
Last Closed: 2015-07-29 05:05:33 UTC
Embargoed:


Attachments (Terms of Use)
QA_console logs where the bug was identified (25.31 KB, application/x-gzip)
2015-06-17 07:56 UTC, Nag Pavan Chilakam
no flags Details
server#1 logs sosreports (7.86 MB, application/x-gzip)
2015-06-17 08:49 UTC, Nag Pavan Chilakam
no flags Details
server#2 logs sosreports (7.86 MB, application/x-gzip)
2015-06-17 08:50 UTC, Nag Pavan Chilakam
no flags Details
QA_console logs where the bug was identified(changed file permissions) (25.31 KB, application/x-gzip)
2015-06-17 08:53 UTC, Nag Pavan Chilakam
no flags Details
QE verified cli log (32.66 KB, text/plain)
2015-07-07 07:56 UTC, Nag Pavan Chilakam
no flags Details
rebalance logs for QE verified (11.40 KB, text/plain)
2015-07-07 07:59 UTC, Nag Pavan Chilakam
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1495 0 normal SHIPPED_LIVE Important: Red Hat Gluster Storage 3.1 update 2015-07-29 08:26:26 UTC

Description Nag Pavan Chilakam 2015-06-17 07:54:26 UTC
Description of problem:
=======================
After created a tiered volume, I have added a couple of files which went to the default hot tier as expected.
I have set only "features.ctr-enabled" to "ON" and no other tiering parameter.
But I noticed that when I don't work on that file for sometime, the file gets demoted to cold tier.
But once a file is demoted to cold tier, it  is not getting promoted at all.
As discussed the default frequency counter is 120.
That means if a file in cold tier is modified(i have touched the file) more than 120 times in the given default promote time, it should get promoted automatically.
But this is not happening at all


Version-Release number of selected component (if applicable):
==============================================================
[root@rhsqa14-vm4 glusterfs]# gluster --version
glusterfs 3.7.1 built on Jun 14 2015 21:55:41
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@rhsqa14-vm4 glusterfs]# rpm -qa|grep gluster
glusterfs-libs-3.7.1-3.el6rhs.x86_64
glusterfs-fuse-3.7.1-3.el6rhs.x86_64
glusterfs-3.7.1-3.el6rhs.x86_64
glusterfs-api-3.7.1-3.el6rhs.x86_64
glusterfs-cli-3.7.1-3.el6rhs.x86_64
glusterfs-geo-replication-3.7.1-3.el6rhs.x86_64
glusterfs-debuginfo-3.7.1-3.el6rhs.x86_64
glusterfs-client-xlators-3.7.1-3.el6rhs.x86_64
glusterfs-server-3.7.1-3.el6rhs.x86_64
glusterfs-rdma-3.7.1-3.el6rhs.x86_64



How reproducible:
================
everytime


Steps to Reproduce:
===================
1.create a tiered volume
2.set only "features.ctr-enabled" to "ON"
3.now create a file which will go to hot tier.
4. Wait for some time while keeping the setup and file idle(say 3 min) and you can see file gets demoted
It can be seen that file is demoted
5. Now touch the file any number of times back to back and it can be seen that file never gets promoted.

Actual behavior:
===============
A file once demoted is not getting promoted at all even on modification any number of times back to back and it can be seen that file never gets promoted.
Also, if we think we must set all tiering parameters(vol set options), for file to get promoted, then in that case file demotion too should happen only when all params are set. But in this case file demote happens but not promoted back


Expected results:
===================
file should get promoted if modified the default number of times or more and the default time period.
Also, if we think we must set all tiering parameters(vol set options), for file to get promoted, then in that case file demotion too should happen only when all params are set. But in this case file demote happens but not promoted back

Additional info:
===============
I have used a regular dist-rep volume and mounted on fuse and nfs simultanteously and while pumping IO (copying a huge file of 4gb), i attached a tier. On this kind of volume i reused for this test case

Comment 2 Nag Pavan Chilakam 2015-06-17 07:56:07 UTC
Created attachment 1039790 [details]
QA_console logs where the bug was identified

Comment 3 Nag Pavan Chilakam 2015-06-17 08:49:28 UTC
Created attachment 1039805 [details]
server#1 logs sosreports

Comment 4 Nag Pavan Chilakam 2015-06-17 08:50:54 UTC
Created attachment 1039806 [details]
server#2 logs sosreports

Comment 5 Nag Pavan Chilakam 2015-06-17 08:51:42 UTC
sos reports attached

Comment 6 Nag Pavan Chilakam 2015-06-17 08:53:23 UTC
Created attachment 1039811 [details]
QA_console logs where the bug was identified(changed file permissions)

Comment 10 Nagaprasad Sathyanarayana 2015-06-24 17:03:52 UTC
http://review.gluster.org/#/c/11334/

Comment 11 Joseph Elwin Fernandes 2015-06-27 15:21:44 UTC
https://code.engineering.redhat.com/gerrit/#/c/51756/

Comment 12 Joseph Elwin Fernandes 2015-06-27 15:24:55 UTC
https://code.engineering.redhat.com/gerrit/#/c/49748/

Comment 13 Nag Pavan Chilakam 2015-07-07 07:53:47 UTC
A File in a simple case is now getting promoted and demoted any number of times.
Hence moving this bug as verified

1)Created a  tier vol
2)set following options
Type: Tier
Volume ID: 232deda8-c5c1-491e-a20d-4d34d3cffff1
Status: Started
Number of Bricks: 6
Transport-type: tcp
Hot Tier :
Hot Tier Type : Distribute
Number of Bricks: 2
Brick1: 10.70.42.203:/rhs/brick3/vol2
Brick2: 10.70.42.73:/rhs/brick3/vol2
Cold Tier:
Cold Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick3: 10.70.42.73:/rhs/brick1/vol2
Brick4: 10.70.42.144:/rhs/brick1/vol2
Brick5: 10.70.42.73:/rhs/brick2/vol2
Brick6: 10.70.42.144:/rhs/brick2/vol2
Options Reconfigured:
cluster.write-freq-threshold: 0
cluster.read-freq-threshold: 0
features.record-counters: off
cluster.tier-demote-frequency: 120
cluster.tier-promote-frequency: 120
features.ctr-enabled: on
performance.quick-read: off
performance.io-cache: off
performance.readdir-ahead: on


3) Now mounted file on nfs
4)created a file
5)waited for some time by keeping file idle
6)the file got demoted after sometime.
7)then touched the file to change ctime
8)now the file got promoted again
did this in a loop twice and promotions and demotions were happening

Hence moving the bug to verified(fixed)

Note: Following were some observations:
1)fusemount
2)when multiple files were played with, some files were not getting migrated. A seperate bug was raised bz#1238944  
3) Didn't play with the volume tier settings and options due to the scope of feature
4)Previously a file demoted was not getting promoted at all, but now the problem perse was fixed hence moving to verified


Sos reports and logs attached




version:
=======
[root@nchilaka-tier01 ~]# gluster --version
glusterfs 3.7.1 built on Jul  2 2015 21:01:51
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@nchilaka-tier01 ~]# rpm -qa|grep gluster
gluster-nagios-common-0.2.0-1.el6rhs.noarch
vdsm-gluster-4.16.20-1.2.el6rhs.noarch
glusterfs-client-xlators-3.7.1-7.el6rhs.x86_64
glusterfs-server-3.7.1-7.el6rhs.x86_64
nfs-ganesha-gluster-2.2.0-3.el6rhs.x86_64
python-gluster-3.7.1-6.el6rhs.x86_64
glusterfs-3.7.1-7.el6rhs.x86_64
glusterfs-api-3.7.1-7.el6rhs.x86_64
glusterfs-cli-3.7.1-7.el6rhs.x86_64
glusterfs-geo-replication-3.7.1-7.el6rhs.x86_64
glusterfs-rdma-3.7.1-7.el6rhs.x86_64
gluster-nagios-addons-0.2.4-2.el6rhs.x86_64
glusterfs-libs-3.7.1-7.el6rhs.x86_64
glusterfs-fuse-3.7.1-7.el6rhs.x86_64
glusterfs-ganesha-3.7.1-7.el6rhs.x86_64

Comment 14 Nag Pavan Chilakam 2015-07-07 07:56:34 UTC
Created attachment 1049126 [details]
QE verified cli log

Comment 15 Nag Pavan Chilakam 2015-07-07 07:59:44 UTC
Created attachment 1049133 [details]
rebalance logs for QE verified

Comment 16 errata-xmlrpc 2015-07-29 05:05:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1495.html


Note You need to log in before you can comment on or make changes to this bug.