Bug 1290424

Summary: Tier Migration not abiding to cluster.tier-max-mb settings
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Nag Pavan Chilakam <nchilaka>
Component: tierAssignee: hari gowtham <hgowtham>
Status: CLOSED WONTFIX QA Contact: Nag Pavan Chilakam <nchilaka>
Severity: urgent Docs Contact:
Priority: urgent    
Version: rhgs-3.1CC: mzywusko, rhs-bugs
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: tier-migration
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-06 17:42:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nag Pavan Chilakam 2015-12-10 13:29:01 UTC
Description of problem:
======================
With the default value of cluster.tier-max-mb being 1000MB(~1GB), there should not be a demote or promote cummilatively more than 1GB in one cycle.
But i see in my case that the migration doens't abide to this. In my case about 5GB got promoted in one cycle(default 120sec)


Version-Release number of selected component (if applicable):
======================
3.7.5-9




Steps to Reproduce:
====================
Was trying to reproduce 1286218
 Used the same steps:
Volume Name: goblin
Type: Tier
Volume ID: 3dbdc745-7306-460f-a4ea-754584cdb4b2
Status: Started
Number of Bricks: 14
Transport-type: tcp
Hot Tier :
Hot Tier Type : Replicate
Number of Bricks: 1 x 2 = 2
Brick1: yarrow:/rhs/brick5/1/brick1007/goblin_hot
Brick2: zod:/rhs/brick5/1/brick1007/goblin_hot
Cold Tier:
Cold Tier Type : Distributed-Disperse
Number of Bricks: 2 x (4 + 2) = 12
Brick3: zod:/rhs/brick5/1/brick1001/goblin
Brick4: yarrow:/rhs/brick5/1/brick1001/goblin
Brick5: zod:/rhs/brick5/1/brick1002/goblin
Brick6: yarrow:/rhs/brick5/1/brick1002/goblin
Brick7: zod:/rhs/brick5/1/brick1003/goblin
Brick8: yarrow:/rhs/brick5/1/brick1003/goblin
Brick9: zod:/rhs/brick5/1/brick1004/goblin
Brick10: yarrow:/rhs/brick5/1/brick1004/goblin
Brick11: zod:/rhs/brick5/1/brick1005/goblin
Brick12: yarrow:/rhs/brick5/1/brick1005/goblin
Brick13: zod:/rhs/brick5/1/brick1006/goblin
Brick14: yarrow:/rhs/brick5/1/brick1006/goblin
Options Reconfigured:
cluster.watermark-hi: 5
cluster.watermark-low: 1
cluster.tier-mode: cache
features.ctr-enabled: on
performance.readdir-ahead: on

[root@zod ~]# gluster v get goblin all|grep max
performance.cache-max-file-size         0                                       
features.trash-max-filesize             5MB                                     
cluster.tier-max-mb                     1000                                    
cluster.tier-max-files                  5000                                    
[root@zod ~]# gluster v info goblin


I saw that more than 4GB got promoted in one cycle


LOGS:
====
=============>Now no files in hot tier<===================================
[root@zod ~]# date
Thu Dec 10 16:41:24 IST 2015
[root@zod ~]# echo "###############################";date;ll /rhs/brick5/1/brick10*/goblin*
###############################
Thu Dec 10 16:41:26 IST 2015
/rhs/brick5/1/brick1001/goblin:
total 1454456
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.5
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb2.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb3.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb4.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1002/goblin:
total 1454456
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.5
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb2.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb3.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb4.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1003/goblin:
total 1454456
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.5
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb2.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb3.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb4.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1004/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1005/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1006/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1007/goblin_hot:
total 0

=============>Heated files by appending a line on mount<=====================
[root@localhost goblin]#  for i in {1..10};do echo "hi" >>bb$i.mp;done
[root@localhost goblin]# for i in {1..10};do echo "hi" >>200mb.$i;done
===============>Migration started<===================
[root@zod ~]# date
Thu Dec 10 16:42:12 IST 2015
[root@zod ~]#  echo "===========Date=====================";date; echo "=============ColdBrick#1 =========" ;  echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1001/goblin/.glusterfs/goblin.db;echo "=============ColdBrick#2 =========" ;  echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1002/goblin/.glusterfs/goblin.db;echo "=============ColdBrick#3 =========" ;  echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1003/goblin/.glusterfs/goblin.db; echo "=============ColdBrick#4 =========" ;  echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1004/goblin/.glusterfs/goblin.db; echo "=============ColdBrick#5 =========" ;  echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1005/goblin/.glusterfs/goblin.db; echo "=============ColdBrick#6 =========" ;  echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1006/goblin/.glusterfs/goblin.db; echo ">>>>>>>>>>>> HOTBRICK#1 <<<<<<<<==";echo "select * from gf_file_tb; select * from gf_flink_tb;" | sqlite3 /rhs/brick5/1/brick1007/goblin_hot/.glusterfs/goblin_hot.db;
===========Date=====================
Thu Dec 10 16:42:17 IST 2015
=============ColdBrick#1 =========
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|1449745840|831288|0|0|1449743351|663109|0|0|0|0
2263d0a2-0023-4d92-8f9e-718f0dbe320a|1449745840|802685|0|0|0|0|0|0|0|0
2e0d0071-575f-42ed-93ff-6bb2e5fc6947|1449745840|822273|0|0|0|0|0|0|0|0
32cac163-4226-48c8-b630-c139b64f53f5|1449745840|850145|0|0|0|0|0|0|0|0
3ab0e5eb-034f-43ce-a905-d984715f7204|1449745840|859126|0|0|0|0|0|0|0|0
5325c521-2efe-40da-861d-6105b061cb1d|1449745840|812233|0|0|1449743466|755595|0|0|0|0
8f2c337f-b045-4f7e-8161-05dab5a47813|1449745840|840311|0|0|0|0|0|0|0|0
e4a30191-6467-4340-aadb-d4210e3a8d8b|0|0|0|0|0|0|0|0|0|0
0c960150-41dc-4dda-8f78-f2d7b7addfbc|1449745842|986801|0|0|0|0|0|0|0|0
fb7bc6e1-9151-4648-a75f-ca6cbac3dcb4|1449745842|949358|0|0|1449743082|106381|0|0|0|0
209858ce-6d95-4f0d-8492-2e17b9fa179d|1449745842|995789|0|0|0|0|0|0|0|0
6a6448fb-26e7-410a-ba0c-e3f38d4b7813|1449745843|5032|0|0|0|0|0|0|0|0
f2068d74-eb8c-4c58-bbcf-7468ec6f626a|1449745843|13727|0|0|0|0|0|0|0|0
9c901f15-70d4-40d7-abbd-51d4a139469d|1449745843|29950|0|0|0|0|0|0|0|0
c0dd7128-b934-42ba-ac4b-aab6944126dc|1449745840|792898|0|0|0|0|0|0|0|0
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|00000000-0000-0000-0000-000000000001|bb5.mp|/bb5.mp|0|0
2263d0a2-0023-4d92-8f9e-718f0dbe320a|00000000-0000-0000-0000-000000000001|bb2.mp|/bb2.mp|0|0
2e0d0071-575f-42ed-93ff-6bb2e5fc6947|00000000-0000-0000-0000-000000000001|bb4.mp|/bb4.mp|0|0
32cac163-4226-48c8-b630-c139b64f53f5|00000000-0000-0000-0000-000000000001|bb7.mp|/bb7.mp|0|0
3ab0e5eb-034f-43ce-a905-d984715f7204|00000000-0000-0000-0000-000000000001|bb8.mp|/bb8.mp|0|0
5325c521-2efe-40da-861d-6105b061cb1d|00000000-0000-0000-0000-000000000001|bb3.mp|/bb3.mp|0|0
8f2c337f-b045-4f7e-8161-05dab5a47813|00000000-0000-0000-0000-000000000001|bb6.mp|/bb6.mp|0|0
e4a30191-6467-4340-aadb-d4210e3a8d8b|00000000-0000-0000-0000-000000000001|onegb.file|/onegb.file|0|0
0c960150-41dc-4dda-8f78-f2d7b7addfbc|00000000-0000-0000-0000-000000000001|200mb.5|/200mb.5|0|0
fb7bc6e1-9151-4648-a75f-ca6cbac3dcb4|00000000-0000-0000-0000-000000000001|200mb.1|/200mb.1|0|0
209858ce-6d95-4f0d-8492-2e17b9fa179d|00000000-0000-0000-0000-000000000001|200mb.6|/200mb.6|0|0
6a6448fb-26e7-410a-ba0c-e3f38d4b7813|00000000-0000-0000-0000-000000000001|200mb.7|/200mb.7|0|0
f2068d74-eb8c-4c58-bbcf-7468ec6f626a|00000000-0000-0000-0000-000000000001|200mb.8|/200mb.8|0|0
9c901f15-70d4-40d7-abbd-51d4a139469d|00000000-0000-0000-0000-000000000001|200mb.10|/200mb.10|0|0
c0dd7128-b934-42ba-ac4b-aab6944126dc|00000000-0000-0000-0000-000000000001|bb1.mp|/bb1.mp|0|0
=============ColdBrick#2 =========
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|1449745840|831423|0|0|1449743351|663662|0|0|0|0
2263d0a2-0023-4d92-8f9e-718f0dbe320a|1449745840|802849|0|0|0|0|0|0|0|0
2e0d0071-575f-42ed-93ff-6bb2e5fc6947|1449745840|822425|0|0|0|0|0|0|0|0
32cac163-4226-48c8-b630-c139b64f53f5|1449745840|850260|0|0|0|0|0|0|0|0
3ab0e5eb-034f-43ce-a905-d984715f7204|1449745840|859312|0|0|0|0|0|0|0|0
5325c521-2efe-40da-861d-6105b061cb1d|1449745840|812189|0|0|1449743466|755682|0|0|0|0
8f2c337f-b045-4f7e-8161-05dab5a47813|1449745840|840375|0|0|0|0|0|0|0|0
e4a30191-6467-4340-aadb-d4210e3a8d8b|0|0|0|0|0|0|0|0|0|0
0c960150-41dc-4dda-8f78-f2d7b7addfbc|1449745842|986938|0|0|0|0|0|0|0|0
fb7bc6e1-9151-4648-a75f-ca6cbac3dcb4|1449745842|949499|0|0|1449743082|106387|0|0|0|0
209858ce-6d95-4f0d-8492-2e17b9fa179d|1449745842|996073|0|0|0|0|0|0|0|0
6a6448fb-26e7-410a-ba0c-e3f38d4b7813|1449745843|5236|0|0|0|0|0|0|0|0
f2068d74-eb8c-4c58-bbcf-7468ec6f626a|1449745843|13818|0|0|0|0|0|0|0|0
9c901f15-70d4-40d7-abbd-51d4a139469d|1449745843|30066|0|0|0|0|0|0|0|0
c0dd7128-b934-42ba-ac4b-aab6944126dc|1449745840|793075|0|0|0|0|0|0|0|0
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|00000000-0000-0000-0000-000000000001|bb5.mp|/bb5.mp|0|0
2263d0a2-0023-4d92-8f9e-718f0dbe320a|00000000-0000-0000-0000-000000000001|bb2.mp|/bb2.mp|0|0
2e0d0071-575f-42ed-93ff-6bb2e5fc6947|00000000-0000-0000-0000-000000000001|bb4.mp|/bb4.mp|0|0
32cac163-4226-48c8-b630-c139b64f53f5|00000000-0000-0000-0000-000000000001|bb7.mp|/bb7.mp|0|0
3ab0e5eb-034f-43ce-a905-d984715f7204|00000000-0000-0000-0000-000000000001|bb8.mp|/bb8.mp|0|0
5325c521-2efe-40da-861d-6105b061cb1d|00000000-0000-0000-0000-000000000001|bb3.mp|/bb3.mp|0|0
8f2c337f-b045-4f7e-8161-05dab5a47813|00000000-0000-0000-0000-000000000001|bb6.mp|/bb6.mp|0|0
e4a30191-6467-4340-aadb-d4210e3a8d8b|00000000-0000-0000-0000-000000000001|onegb.file|/onegb.file|0|0
0c960150-41dc-4dda-8f78-f2d7b7addfbc|00000000-0000-0000-0000-000000000001|200mb.5|/200mb.5|0|0
fb7bc6e1-9151-4648-a75f-ca6cbac3dcb4|00000000-0000-0000-0000-000000000001|200mb.1|/200mb.1|0|0
209858ce-6d95-4f0d-8492-2e17b9fa179d|00000000-0000-0000-0000-000000000001|200mb.6|/200mb.6|0|0
6a6448fb-26e7-410a-ba0c-e3f38d4b7813|00000000-0000-0000-0000-000000000001|200mb.7|/200mb.7|0|0
f2068d74-eb8c-4c58-bbcf-7468ec6f626a|00000000-0000-0000-0000-000000000001|200mb.8|/200mb.8|0|0
9c901f15-70d4-40d7-abbd-51d4a139469d|00000000-0000-0000-0000-000000000001|200mb.10|/200mb.10|0|0
c0dd7128-b934-42ba-ac4b-aab6944126dc|00000000-0000-0000-0000-000000000001|bb1.mp|/bb1.mp|0|0
=============ColdBrick#3 =========
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|1449745840|831533|0|0|0|0|0|0|0|0
2263d0a2-0023-4d92-8f9e-718f0dbe320a|1449745840|802940|0|0|0|0|0|0|0|0
2e0d0071-575f-42ed-93ff-6bb2e5fc6947|1449745840|822537|0|0|0|0|0|0|0|0
32cac163-4226-48c8-b630-c139b64f53f5|1449745840|850398|0|0|0|0|0|0|0|0
3ab0e5eb-034f-43ce-a905-d984715f7204|1449745840|859419|0|0|0|0|0|0|0|0
5325c521-2efe-40da-861d-6105b061cb1d|1449745840|812267|0|0|1449743466|750557|0|0|0|0
8f2c337f-b045-4f7e-8161-05dab5a47813|1449745840|840485|0|0|0|0|0|0|0|0
e4a30191-6467-4340-aadb-d4210e3a8d8b|0|0|0|0|0|0|0|0|0|0
0c960150-41dc-4dda-8f78-f2d7b7addfbc|1449745842|987005|0|0|0|0|0|0|0|0
fb7bc6e1-9151-4648-a75f-ca6cbac3dcb4|1449745842|949626|0|0|1449743082|99396|0|0|0|0
209858ce-6d95-4f0d-8492-2e17b9fa179d|1449745842|996185|0|0|0|0|0|0|0|0
6a6448fb-26e7-410a-ba0c-e3f38d4b7813|1449745843|5302|0|0|0|0|0|0|0|0
f2068d74-eb8c-4c58-bbcf-7468ec6f626a|1449745843|13996|0|0|0|0|0|0|0|0
9c901f15-70d4-40d7-abbd-51d4a139469d|1449745843|30199|0|0|0|0|0|0|0|0
c0dd7128-b934-42ba-ac4b-aab6944126dc|1449745840|793196|0|0|0|0|0|0|0|0
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|00000000-0000-0000-0000-000000000001|bb5.mp|/bb5.mp|0|0
2263d0a2-0023-4d92-8f9e-718f0dbe320a|00000000-0000-0000-0000-000000000001|bb2.mp|/bb2.mp|0|0
2e0d0071-575f-42ed-93ff-6bb2e5fc6947|00000000-0000-0000-0000-000000000001|bb4.mp|/bb4.mp|0|0
32cac163-4226-48c8-b630-c139b64f53f5|00000000-0000-0000-0000-000000000001|bb7.mp|/bb7.mp|0|0
3ab0e5eb-034f-43ce-a905-d984715f7204|00000000-0000-0000-0000-000000000001|bb8.mp|/bb8.mp|0|0
5325c521-2efe-40da-861d-6105b061cb1d|00000000-0000-0000-0000-000000000001|bb3.mp|/bb3.mp|0|0
8f2c337f-b045-4f7e-8161-05dab5a47813|00000000-0000-0000-0000-000000000001|bb6.mp|/bb6.mp|0|0
e4a30191-6467-4340-aadb-d4210e3a8d8b|00000000-0000-0000-0000-000000000001|onegb.file|/onegb.file|0|0
0c960150-41dc-4dda-8f78-f2d7b7addfbc|00000000-0000-0000-0000-000000000001|200mb.5|/200mb.5|0|0
fb7bc6e1-9151-4648-a75f-ca6cbac3dcb4|00000000-0000-0000-0000-000000000001|200mb.1|/200mb.1|0|0
209858ce-6d95-4f0d-8492-2e17b9fa179d|00000000-0000-0000-0000-000000000001|200mb.6|/200mb.6|0|0
6a6448fb-26e7-410a-ba0c-e3f38d4b7813|00000000-0000-0000-0000-000000000001|200mb.7|/200mb.7|0|0
f2068d74-eb8c-4c58-bbcf-7468ec6f626a|00000000-0000-0000-0000-000000000001|200mb.8|/200mb.8|0|0
9c901f15-70d4-40d7-abbd-51d4a139469d|00000000-0000-0000-0000-000000000001|200mb.10|/200mb.10|0|0
c0dd7128-b934-42ba-ac4b-aab6944126dc|00000000-0000-0000-0000-000000000001|bb1.mp|/bb1.mp|0|0
=============ColdBrick#4 =========
62a92304-43c5-4d4c-8fe5-2bc8359e1de4|1449745840|881354|0|0|0|0|0|0|0|0
7e84bbf7-c1f1-4ec3-9b8b-e26081470242|0|0|0|0|0|0|0|0|0|0
aa4103ec-8cc4-40d4-ac8b-5e69f2b0c12e|1449745840|869089|0|0|0|0|0|0|0|0
8b5b9ce3-d49e-42e6-a583-4393968839cc|1449745842|977281|0|0|0|0|0|0|0|0
bd05d6fa-cf37-4b60-9aa3-7dd9511f1680|1449745842|958914|0|0|0|0|0|0|0|0
fae780c3-080b-4259-be19-a229359881d5|1449745842|968342|0|0|0|0|0|0|0|0
06bd9f0b-a6df-4dbb-8da2-d240ed0b94aa|1449745843|21763|0|0|0|0|0|0|0|0
62a92304-43c5-4d4c-8fe5-2bc8359e1de4|00000000-0000-0000-0000-000000000001|bb10.mp|/bb10.mp|0|0
7e84bbf7-c1f1-4ec3-9b8b-e26081470242|00000000-0000-0000-0000-000000000001|bb.mp|/bb.mp|0|0
aa4103ec-8cc4-40d4-ac8b-5e69f2b0c12e|00000000-0000-0000-0000-000000000001|bb9.mp|/bb9.mp|0|0
8b5b9ce3-d49e-42e6-a583-4393968839cc|00000000-0000-0000-0000-000000000001|200mb.4|/200mb.4|0|0
bd05d6fa-cf37-4b60-9aa3-7dd9511f1680|00000000-0000-0000-0000-000000000001|200mb.2|/200mb.2|0|0
fae780c3-080b-4259-be19-a229359881d5|00000000-0000-0000-0000-000000000001|200mb.3|/200mb.3|0|0
06bd9f0b-a6df-4dbb-8da2-d240ed0b94aa|00000000-0000-0000-0000-000000000001|200mb.9|/200mb.9|0|0
=============ColdBrick#5 =========
62a92304-43c5-4d4c-8fe5-2bc8359e1de4|1449745840|881599|0|0|0|0|0|0|0|0
7e84bbf7-c1f1-4ec3-9b8b-e26081470242|0|0|0|0|0|0|0|0|0|0
aa4103ec-8cc4-40d4-ac8b-5e69f2b0c12e|1449745840|869157|0|0|0|0|0|0|0|0
8b5b9ce3-d49e-42e6-a583-4393968839cc|1449745842|977378|0|0|0|0|0|0|0|0
bd05d6fa-cf37-4b60-9aa3-7dd9511f1680|1449745842|959055|0|0|0|0|0|0|0|0
fae780c3-080b-4259-be19-a229359881d5|1449745842|968426|0|0|0|0|0|0|0|0
06bd9f0b-a6df-4dbb-8da2-d240ed0b94aa|1449745843|21955|0|0|0|0|0|0|0|0
62a92304-43c5-4d4c-8fe5-2bc8359e1de4|00000000-0000-0000-0000-000000000001|bb10.mp|/bb10.mp|0|0
7e84bbf7-c1f1-4ec3-9b8b-e26081470242|00000000-0000-0000-0000-000000000001|bb.mp|/bb.mp|0|0
aa4103ec-8cc4-40d4-ac8b-5e69f2b0c12e|00000000-0000-0000-0000-000000000001|bb9.mp|/bb9.mp|0|0
8b5b9ce3-d49e-42e6-a583-4393968839cc|00000000-0000-0000-0000-000000000001|200mb.4|/200mb.4|0|0
bd05d6fa-cf37-4b60-9aa3-7dd9511f1680|00000000-0000-0000-0000-000000000001|200mb.2|/200mb.2|0|0
fae780c3-080b-4259-be19-a229359881d5|00000000-0000-0000-0000-000000000001|200mb.3|/200mb.3|0|0
06bd9f0b-a6df-4dbb-8da2-d240ed0b94aa|00000000-0000-0000-0000-000000000001|200mb.9|/200mb.9|0|0
=============ColdBrick#6 =========
62a92304-43c5-4d4c-8fe5-2bc8359e1de4|1449745840|881796|0|0|0|0|0|0|0|0
7e84bbf7-c1f1-4ec3-9b8b-e26081470242|0|0|0|0|0|0|0|0|0|0
aa4103ec-8cc4-40d4-ac8b-5e69f2b0c12e|1449745840|869289|0|0|0|0|0|0|0|0
8b5b9ce3-d49e-42e6-a583-4393968839cc|1449745842|977438|0|0|0|0|0|0|0|0
bd05d6fa-cf37-4b60-9aa3-7dd9511f1680|1449745842|959216|0|0|0|0|0|0|0|0
fae780c3-080b-4259-be19-a229359881d5|1449745842|968681|0|0|0|0|0|0|0|0
06bd9f0b-a6df-4dbb-8da2-d240ed0b94aa|1449745843|22077|0|0|0|0|0|0|0|0
62a92304-43c5-4d4c-8fe5-2bc8359e1de4|00000000-0000-0000-0000-000000000001|bb10.mp|/bb10.mp|0|0
7e84bbf7-c1f1-4ec3-9b8b-e26081470242|00000000-0000-0000-0000-000000000001|bb.mp|/bb.mp|0|0
aa4103ec-8cc4-40d4-ac8b-5e69f2b0c12e|00000000-0000-0000-0000-000000000001|bb9.mp|/bb9.mp|0|0
8b5b9ce3-d49e-42e6-a583-4393968839cc|00000000-0000-0000-0000-000000000001|200mb.4|/200mb.4|0|0
bd05d6fa-cf37-4b60-9aa3-7dd9511f1680|00000000-0000-0000-0000-000000000001|200mb.2|/200mb.2|0|0
fae780c3-080b-4259-be19-a229359881d5|00000000-0000-0000-0000-000000000001|200mb.3|/200mb.3|0|0
06bd9f0b-a6df-4dbb-8da2-d240ed0b94aa|00000000-0000-0000-0000-000000000001|200mb.9|/200mb.9|0|0
>>>>>>>>>>>> HOTBRICK#1 <<<<<<<<==
0c960150-41dc-4dda-8f78-f2d7b7addfbc|1449745921|292783|0|0|0|0|0|0|1|1
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|1449745929|221326|0|0|0|0|0|0|1|1
0c960150-41dc-4dda-8f78-f2d7b7addfbc|00000000-0000-0000-0000-000000000001|200mb.5|/200mb.5|0|0
108141b2-bf7f-4d48-8c4a-8efa2eca7cc6|00000000-0000-0000-0000-000000000001|bb5.mp|/bb5.mp|0|0
[root@zod ~]# echo "###############################";date;ll /rhs/brick5/1/brick10*/goblin*
###############################
Thu Dec 10 16:42:21 IST 2015
/rhs/brick5/1/brick1001/goblin:
total 1404452
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
---------T. 2 root root         0 Dec 10 16:42 200mb.5
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb2.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb3.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb4.mp
-rw---S--T. 2 root root 183983616 Dec 10 16:40 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1002/goblin:
total 1404452
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
---------T. 2 root root         0 Dec 10 16:42 200mb.5
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb2.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb3.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb4.mp
-rw---S--T. 2 root root 183983616 Dec 10 16:40 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1003/goblin:
total 1404452
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
---------T. 2 root root         0 Dec 10 16:42 200mb.5
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb2.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb3.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb4.mp
-rw---S--T. 2 root root 183983616 Dec 10 16:40 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
-rw-------. 2 root root       512 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1004/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1005/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1006/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1007/goblin_hot:
total 518604
-rw-r--r--. 2 root root 204800006 Dec 10 16:40 200mb.5
---------T. 2 root root 735933672 Dec 10 16:42 bb5.mp
[root@zod ~]# echo "###############################";date;ll /rhs/brick5/1/brick10*/goblin*


=============>almost end of cycle, more than 1GB got promoted<=========
[root@zod ~]# du -sh /rhs/brick5/1/brick1007/goblin_hot/*
196M	/rhs/brick5/1/brick1007/goblin_hot/200mb.5
196M	/rhs/brick5/1/brick1007/goblin_hot/200mb.6
702M	/rhs/brick5/1/brick1007/goblin_hot/bb2.mp
499M	/rhs/brick5/1/brick1007/goblin_hot/bb3.mp
8.0K	/rhs/brick5/1/brick1007/goblin_hot/bb4.mp
702M	/rhs/brick5/1/brick1007/goblin_hot/bb5.mp
8.0K	/rhs/brick5/1/brick1007/goblin_hot/bb7.mp
702M	/rhs/brick5/1/brick1007/goblin_hot/bb8.mp
[root@zod ~]# echo "###############################";date;ll /rhs/brick5/1/brick10*/goblin*
###############################
Thu Dec 10 16:44:04 IST 2015
/rhs/brick5/1/brick1001/goblin:
total 815424
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
---------T. 2 root root         0 Dec 10 16:42 200mb.5
---------T. 2 root root         0 Dec 10 16:42 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
---------T. 2 root root         0 Dec 10 16:43 bb2.mp
-rw---S--T. 2 root root 183983616 Dec 10 16:40 bb3.mp
---------T. 2 root root         0 Dec 10 16:43 bb4.mp
---------T. 2 root root         0 Dec 10 16:42 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
---------T. 2 root root         0 Dec 10 16:43 bb7.mp
---------T. 2 root root         0 Dec 10 16:43 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1002/goblin:
total 815424
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
---------T. 2 root root         0 Dec 10 16:42 200mb.5
---------T. 2 root root         0 Dec 10 16:42 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
---------T. 2 root root         0 Dec 10 16:43 bb2.mp
-rw---S--T. 2 root root 183983616 Dec 10 16:40 bb3.mp
---------T. 2 root root         0 Dec 10 16:43 bb4.mp
---------T. 2 root root         0 Dec 10 16:42 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
---------T. 2 root root         0 Dec 10 16:43 bb7.mp
---------T. 2 root root         0 Dec 10 16:43 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1003/goblin:
total 815424
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.1
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.10
---------T. 2 root root         0 Dec 10 16:42 200mb.5
---------T. 2 root root         0 Dec 10 16:42 200mb.6
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.7
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.8
-rw-r--r--. 2 root root       512 Dec 10 16:40 bb1.mp
---------T. 2 root root         0 Dec 10 16:43 bb2.mp
-rw---S--T. 2 root root 183983616 Dec 10 16:40 bb3.mp
---------T. 2 root root         0 Dec 10 16:43 bb4.mp
---------T. 2 root root         0 Dec 10 16:42 bb5.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb6.mp
---------T. 2 root root         0 Dec 10 16:43 bb7.mp
---------T. 2 root root         0 Dec 10 16:43 bb8.mp
-rw-r--r--. 2 root root 262144000 Dec 10 15:44 onegb.file

/rhs/brick5/1/brick1004/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1005/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1006/goblin:
total 696156
-rw-r--r--. 2 root root   7266816 Dec 10 16:40 200mb.2
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.3
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.4
-rw-r--r--. 2 root root  51200512 Dec 10 16:40 200mb.9
-rw-------. 2 root root 183983616 Dec 10 16:40 bb10.mp
-rw-------. 2 root root 183983616 Dec 10 16:40 bb9.mp
-rw-------. 2 root root 183983616 Dec 10 15:15 bb.mp

/rhs/brick5/1/brick1007/goblin_hot:
total 3132880
-rw-r--r--. 2 root root 204800006 Dec 10 16:40 200mb.5
-rw-r--r--. 2 root root 204800006 Dec 10 16:40 200mb.6
-rw-------. 2 root root 735933672 Dec 10 16:40 bb2.mp
---------T. 2 root root 735933672 Dec 10 16:44 bb3.mp
-rw-------. 2 root root         9 Dec 10 16:40 bb4.mp
-rw-------. 2 root root 735933672 Dec 10 16:40 bb5.mp
-rw-------. 2 root root         9 Dec 10 16:40 bb7.mp
-rw-------. 2 root root 735933672 Dec 10 16:40 bb8.mp
[root@zod ~]# date
Thu Dec 10 16:44:05 IST 2015
[root@zod ~]# #du -sh /rhs/brick5/1/brick1007/goblin_hot[root@zod ~]#

Comment 6 Shyamsundar 2018-02-06 17:42:59 UTC
Thank you for your bug report.

We are no longer working on any improvements for Tier. This bug will be set to CLOSED WONTFIX to reflect this. Please reopen if the rfe is deemed critical.