Bug 1275998

Summary: Data Tiering: "ls" count taking link files and promote/demote files into consideration both on fuse and nfs mount
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Nag Pavan Chilakam <nchilaka>
Component: tierAssignee: Dan Lambright <dlambrig>
Status: CLOSED ERRATA QA Contact: Nag Pavan Chilakam <nchilaka>
Severity: high Docs Contact:
Priority: urgent    
Version: rhgs-3.1CC: asrivast, byarlaga, dlambrig, jbyers, rhinduja, rhs-bugs, sankarshan, sarumuga, storage-qa-internal
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 3.1.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.7.5-7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1281598 (view as bug list) Environment:
Last Closed: 2016-03-01 05:47:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1238549, 1260783, 1260923, 1278384, 1281598, 1283923    

Description Nag Pavan Chilakam 2015-10-28 10:52:03 UTC 
I created about 24 files on the mount point and while for some files link to files got created, and some are being promoted/demoted, I issued a ls |wc -l
to see the count and found anomolies.
I see that it show 28 as the count instead of 24.

For eg: On the backend bricks I see below files:
[root@zod ~]# ll /rhs/brick*/rosa*|grep ^-
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file18
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file19
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file24
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file25
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file28
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file3
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file21
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file26
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file27
-rw-r--r--. 2 root root 108265472 Oct 28  2015 file29
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file1
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file10
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file11
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file12
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file13
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file16
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file17
---------T. 2 root root         0 Oct 28 16:15 file18
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file2
---------T. 2 root root         0 Oct 28 16:16 file24
-rw-r--r--. 2 root root 186283008 Oct 28  2015 file4
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file14
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file15
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file20
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file22
-rw-r--r--. 2 root root 716800000 Oct 28  2015 file23
[root@zod ~]# ll /rhs/brick*/rosa*|grep ^-|wc -l
27


But on mount point I am supposed to see only 24 files(excludes file promoted/demotes and link file ), but below is what i See.


[root@rhel7-autofuseclient rosa]# ls -lart|wc -l
28



Refer to bug 1271947  also for more details
Comment 2 Rahul Hinduja 2015-11-02 14:24:53 UTC 
This bug hits the geo-replication + tiering testing. 

Testing of geo-replication revolves around the following steps:

1. Create Set of files/directories in numbers on master volume
2. Let those files be synced to slave. 
3. Calculate checksum on master volume and slave volume. 

Because of this bug, everytime the number of regular files used for checksum calculation is different. Hence the checksum never matches between Master and Slave. 

Whole manual and automation testing fails because the checksum doesn't match between master and slave volume.
Comment 3 Rahul Hinduja 2015-11-02 14:31:39 UTC 
For reference:
==============

[root@mia scripts]# ./arequal-checksum -i .trashcan -p /mnt/master_fuse/ 

Entry counts
Regular files   : 11235
Directories     : 2366
Symbolic links  : 860
Other           : 0
Total           : 14461

Metadata checksums
Regular files   : 4bd293
Directories     : 3e9
Symbolic links  : 3e9
Other           : 3e9

Checksums
Regular files   : 13e50308e68df5d09cd4bc3d75bb45e1
Directories     : 7c256c32042f3e6c
Symbolic links  : 0
Other           : 0
Total           : f314d30797198e5d
[root@mia scripts]# ./arequal-checksum -i .trashcan -p /mnt/master_fuse/ 

Entry counts
Regular files   : 11232
Directories     : 2366
Symbolic links  : 860
Other           : 0
Total           : 14458

Metadata checksums
Regular files   : 36be
Directories     : 3e9
Symbolic links  : 3e9
Other           : 3e9

Checksums
Regular files   : bdcdf8be9ea83330c708b83300f2b4f8
Directories     : 5d204e5126556265
Symbolic links  : 0
Other           : 0
Total           : 27e50edcb80fe5ad
[root@mia scripts]# 


Whereas actual should be as seen after disabling ctr:
=====================================================


[root@mia scripts]# ./arequal-checksum -i .trashcan -p /mnt/master_fuse/ 

Entry counts
Regular files   : 11231
Directories     : 2366
Symbolic links  : 860
Other           : 0
Total           : 14457

Metadata checksums
Regular files   : 48a15a
Directories     : 3e9
Symbolic links  : 3e9
Other           : 3e9

Checksums
Regular files   : 50999f1e3b154dcddfdd84fc7e55391e
Directories     : 1d2d4910335e0e6b
Symbolic links  : 0
Other           : 0
Total           : 926952f2761e7ab8
[root@mia scripts]#
Comment 5 Saravanakumar 2015-11-19 11:50:31 UTC 
Updating my observation:

ls listing in both master and slave looks fine.

I can observe another issue:
If I try setting promotion time (with less 
value) than demotion, I don't see promotion happening.

Please refer "Part 2" section below.

=================================================================
[root@gfvm3 ~]# ~/start_tier.sh

++ glusterd -LDEBUG
++ gluster volume create tiervol replica 2 gfvm3:/opt/volume_test/tier_vol/b1_1 gfvm3:/opt/volume_test/tier_vol/b1_2 gfvm3:/opt/volume_test/tier_vol/b2_1 gfvm3://opt/volume_test/tier_vol/b2_2 gfvm3:/opt/volume_test/tier_vol/b3_1 gfvm3:/opt/volume_test/tier_vol/b3_2 force
volume create: tiervol: success: please start the volume to access data
++ gluster volume start tiervol
volume start: tiervol: success
++ gluster volume info tiervol
 
Volume Name: tiervol
Type: Distributed-Replicate
Volume ID: a2065059-e8cf-48a5-a1f8-3c607398b7ef
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: gfvm3:/opt/volume_test/tier_vol/b1_1
Brick2: gfvm3:/opt/volume_test/tier_vol/b1_2
Brick3: gfvm3:/opt/volume_test/tier_vol/b2_1
Brick4: gfvm3:/opt/volume_test/tier_vol/b2_2
Brick5: gfvm3:/opt/volume_test/tier_vol/b3_1
Brick6: gfvm3:/opt/volume_test/tier_vol/b3_2
Options Reconfigured:
performance.readdir-ahead: on
++ gluster volume status tiervol
Status of volume: tiervol
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick gfvm3:/opt/volume_test/tier_vol/b1_1  49164     0          Y       18992
Brick gfvm3:/opt/volume_test/tier_vol/b1_2  49165     0          Y       19011
Brick gfvm3:/opt/volume_test/tier_vol/b2_1  49166     0          Y       19030
Brick gfvm3:/opt/volume_test/tier_vol/b2_2  49167     0          Y       19049
Brick gfvm3:/opt/volume_test/tier_vol/b3_1  49168     0          Y       19068
Brick gfvm3:/opt/volume_test/tier_vol/b3_2  49169     0          Y       19087
NFS Server on localhost                     N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        N       N/A  
 
Task Status of Volume tiervol
------------------------------------------------------------------------------
There are no active volume tasks
 
++ echo y
++ gluster volume attach-tier tiervol replica 2 gfvm3:/opt/volume_test/tier_vol/b4_1 gfvm3:/opt/volume_test/tier_vol/b4_2 gfvm3:/opt/volume_test/tier_vol/b5_1 gfvm3:/opt/volume_test/tier_vol/b5_2 force
volume attach-tier: success
Tiering Migration Functionality: tiervol: success: Attach tier is successful on tiervol. use tier status to check the status.
ID: eeea884c-2996-4798-a43f-fc34aa55048a

++ gluster volume info tiervol
 
Volume Name: tiervol
Type: Tier
Volume ID: a2065059-e8cf-48a5-a1f8-3c607398b7ef
Status: Started
Number of Bricks: 10
Transport-type: tcp
Hot Tier :
Hot Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick1: gfvm3:/opt/volume_test/tier_vol/b5_2
Brick2: gfvm3:/opt/volume_test/tier_vol/b5_1
Brick3: gfvm3:/opt/volume_test/tier_vol/b4_2
Brick4: gfvm3:/opt/volume_test/tier_vol/b4_1
Cold Tier:
Cold Tier Type : Distributed-Replicate
Number of Bricks: 3 x 2 = 6
Brick5: gfvm3:/opt/volume_test/tier_vol/b1_1
Brick6: gfvm3:/opt/volume_test/tier_vol/b1_2
Brick7: gfvm3:/opt/volume_test/tier_vol/b2_1
Brick8: gfvm3:/opt/volume_test/tier_vol/b2_2
Brick9: gfvm3:/opt/volume_test/tier_vol/b3_1
Brick10: gfvm3:/opt/volume_test/tier_vol/b3_2
Options Reconfigured:
features.ctr-enabled: on
performance.readdir-ahead: on
++ gluster volume status tiervol
Status of volume: tiervol
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Hot Bricks:
Brick gfvm3:/opt/volume_test/tier_vol/b5_2  49173     0          Y       19209
Brick gfvm3:/opt/volume_test/tier_vol/b5_1  49172     0          Y       19190
Brick gfvm3:/opt/volume_test/tier_vol/b4_2  49171     0          Y       19171
Brick gfvm3:/opt/volume_test/tier_vol/b4_1  49170     0          Y       19152
Cold Bricks:
Brick gfvm3:/opt/volume_test/tier_vol/b1_1  49164     0          Y       18992
Brick gfvm3:/opt/volume_test/tier_vol/b1_2  49165     0          Y       19011
Brick gfvm3:/opt/volume_test/tier_vol/b2_1  49166     0          Y       19030
Brick gfvm3:/opt/volume_test/tier_vol/b2_2  49167     0          Y       19049
Brick gfvm3:/opt/volume_test/tier_vol/b3_1  49168     0          Y       19068
Brick gfvm3:/opt/volume_test/tier_vol/b3_2  49169     0          Y       19087
NFS Server on localhost                     N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       19237
 
Task Status of Volume tiervol
------------------------------------------------------------------------------
Task                 : Tier migration      
ID                   : eeea884c-2996-4798-a43f-fc34aa55048a
Status               : in progress         
 
++ gluster volume create tv2 gfvm3:/opt/volume_test/tv_2/b1 gfvm3:/opt/volume_test/tv_2/b2 force
volume create: tv2: success: please start the volume to access data
++ gluster volume start tv2
volume start: tv2: success
++ cd /root
++ mkdir -p /mnt/tierd_volume/
++ mount -t glusterfs gfvm3:/tiervol /mnt/tierd_volume/
++ sync
++ mkdir -p /mnt/slave/
++ mount -t glusterfs gfvm3:/tv2 /mnt/slave/
++ sync
++ sleep 1
++ cd -
/root
++ gluster volume set tiervol features.ctr-enabled on
volume set: success
++ gluster system:: execute gsec_create
Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub
++ gluster volume geo-replication tiervol gfvm3::tv2 create push-pem force
Creating geo-replication session between tiervol & gfvm3::tv2 has been successful
++ gluster volume geo-replication tiervol gfvm3::tv2 start
Starting geo-replication session between tiervol & gfvm3::tv2 has been successful
++ gluster volume geo-replication tiervol gfvm3::tv2 status
 
MASTER NODE    MASTER VOL    MASTER BRICK                      SLAVE USER    SLAVE         SLAVE NODE    STATUS     CRAWL STATUS    LAST_SYNCED          
----------------------------------------------------------------------------------------------------------------------------------------------
gfvm3          tiervol       /opt/volume_test/tier_vol/b5_2    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b5_1    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b4_2    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b4_1    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b1_1    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b1_2    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b2_1    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b2_2    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b3_1    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
gfvm3          tiervol       /opt/volume_test/tier_vol/b3_2    root          gfvm3::tv2    N/A           Created    N/A             N/A                  
++ echo -e ''

[root@gfvm3 ~]#  gluster volume geo-replication tiervol gfvm3::tv2 status

MASTER NODE    MASTER VOL    MASTER BRICK                      SLAVE USER    SLAVE         SLAVE NODE    STATUS    CRAWL STATUS       LAST_SYNCED                  
--------------------------------------------------------------------------------------------------------------------------------------------------------
gfvm3          tiervol       /opt/volume_test/tier_vol/b5_2    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b5_1    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b4_2    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b4_1    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b1_1    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b1_2    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b2_1    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b2_2    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b3_1    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
gfvm3          tiervol       /opt/volume_test/tier_vol/b3_2    root          gfvm3::tv2    gfvm3         Active    Changelog Crawl    2015-11-16 16:21:50          
[root@gfvm3 ~]#
[root@gfvm3 ~]#
[root@gfvm3 ~]# cd /mnt/tierd_volume/
[root@gfvm3 tierd_volume]# ls
[root@gfvm3 tierd_volume]# touch file{1..100}
[root@gfvm3 tierd_volume]# ls -l
total 0
-rw-r--r-- 1 root root 0 Nov 16 16:22 file1
-rw-r--r-- 1 root root 0 Nov 16 16:22 file10
-rw-r--r-- 1 root root 0 Nov 16 16:22 file100
-rw-r--r-- 1 root root 0 Nov 16 16:22 file11
-rw-r--r-- 1 root root 0 Nov 16 16:22 file12
-rw-r--r-- 1 root root 0 Nov 16 16:22 file13
-rw-r--r-- 1 root root 0 Nov 16 16:22 file14
-rw-r--r-- 1 root root 0 Nov 16 16:22 file15
-rw-r--r-- 1 root root 0 Nov 16 16:22 file16
-rw-r--r-- 1 root root 0 Nov 16 16:22 file17
-rw-r--r-- 1 root root 0 Nov 16 16:22 file18
-rw-r--r-- 1 root root 0 Nov 16 16:22 file19
-rw-r--r-- 1 root root 0 Nov 16 16:22 file2
-rw-r--r-- 1 root root 0 Nov 16 16:22 file20
-rw-r--r-- 1 root root 0 Nov 16 16:22 file21
-rw-r--r-- 1 root root 0 Nov 16 16:22 file22
-rw-r--r-- 1 root root 0 Nov 16 16:22 file23
-rw-r--r-- 1 root root 0 Nov 16 16:22 file24
-rw-r--r-- 1 root root 0 Nov 16 16:22 file25
-rw-r--r-- 1 root root 0 Nov 16 16:22 file26
-rw-r--r-- 1 root root 0 Nov 16 16:22 file27
-rw-r--r-- 1 root root 0 Nov 16 16:22 file28
-rw-r--r-- 1 root root 0 Nov 16 16:22 file29
-rw-r--r-- 1 root root 0 Nov 16 16:22 file3
-rw-r--r-- 1 root root 0 Nov 16 16:22 file30
-rw-r--r-- 1 root root 0 Nov 16 16:22 file31
-rw-r--r-- 1 root root 0 Nov 16 16:22 file32
-rw-r--r-- 1 root root 0 Nov 16 16:22 file33
-rw-r--r-- 1 root root 0 Nov 16 16:22 file34
-rw-r--r-- 1 root root 0 Nov 16 16:22 file35
-rw-r--r-- 1 root root 0 Nov 16 16:22 file36
-rw-r--r-- 1 root root 0 Nov 16 16:22 file37
-rw-r--r-- 1 root root 0 Nov 16 16:22 file38
-rw-r--r-- 1 root root 0 Nov 16 16:22 file39
-rw-r--r-- 1 root root 0 Nov 16 16:22 file4
-rw-r--r-- 1 root root 0 Nov 16 16:22 file40
-rw-r--r-- 1 root root 0 Nov 16 16:22 file41
-rw-r--r-- 1 root root 0 Nov 16 16:22 file42
-rw-r--r-- 1 root root 0 Nov 16 16:22 file43
-rw-r--r-- 1 root root 0 Nov 16 16:22 file44
-rw-r--r-- 1 root root 0 Nov 16 16:22 file45
-rw-r--r-- 1 root root 0 Nov 16 16:22 file46
-rw-r--r-- 1 root root 0 Nov 16 16:22 file47
-rw-r--r-- 1 root root 0 Nov 16 16:22 file48
-rw-r--r-- 1 root root 0 Nov 16 16:22 file49
-rw-r--r-- 1 root root 0 Nov 16 16:22 file5
-rw-r--r-- 1 root root 0 Nov 16 16:22 file50
-rw-r--r-- 1 root root 0 Nov 16 16:22 file51
-rw-r--r-- 1 root root 0 Nov 16 16:22 file52
-rw-r--r-- 1 root root 0 Nov 16 16:22 file53
-rw-r--r-- 1 root root 0 Nov 16 16:22 file54
-rw-r--r-- 1 root root 0 Nov 16 16:22 file55
-rw-r--r-- 1 root root 0 Nov 16 16:22 file56
-rw-r--r-- 1 root root 0 Nov 16 16:22 file57
-rw-r--r-- 1 root root 0 Nov 16 16:22 file58
-rw-r--r-- 1 root root 0 Nov 16 16:22 file59
-rw-r--r-- 1 root root 0 Nov 16 16:22 file6
-rw-r--r-- 1 root root 0 Nov 16 16:22 file60
-rw-r--r-- 1 root root 0 Nov 16 16:22 file61
-rw-r--r-- 1 root root 0 Nov 16 16:22 file62
-rw-r--r-- 1 root root 0 Nov 16 16:22 file63
-rw-r--r-- 1 root root 0 Nov 16 16:22 file64
-rw-r--r-- 1 root root 0 Nov 16 16:22 file65
-rw-r--r-- 1 root root 0 Nov 16 16:22 file66
-rw-r--r-- 1 root root 0 Nov 16 16:22 file67
-rw-r--r-- 1 root root 0 Nov 16 16:22 file68
-rw-r--r-- 1 root root 0 Nov 16 16:22 file69
-rw-r--r-- 1 root root 0 Nov 16 16:22 file7
-rw-r--r-- 1 root root 0 Nov 16 16:22 file70
-rw-r--r-- 1 root root 0 Nov 16 16:22 file71
-rw-r--r-- 1 root root 0 Nov 16 16:22 file72
-rw-r--r-- 1 root root 0 Nov 16 16:22 file73
-rw-r--r-- 1 root root 0 Nov 16 16:22 file74
-rw-r--r-- 1 root root 0 Nov 16 16:22 file75
-rw-r--r-- 1 root root 0 Nov 16 16:22 file76
-rw-r--r-- 1 root root 0 Nov 16 16:22 file77
-rw-r--r-- 1 root root 0 Nov 16 16:22 file78
-rw-r--r-- 1 root root 0 Nov 16 16:22 file79
-rw-r--r-- 1 root root 0 Nov 16 16:22 file8
-rw-r--r-- 1 root root 0 Nov 16 16:22 file80
-rw-r--r-- 1 root root 0 Nov 16 16:22 file81
-rw-r--r-- 1 root root 0 Nov 16 16:22 file82
-rw-r--r-- 1 root root 0 Nov 16 16:22 file83
-rw-r--r-- 1 root root 0 Nov 16 16:22 file84
-rw-r--r-- 1 root root 0 Nov 16 16:22 file85
-rw-r--r-- 1 root root 0 Nov 16 16:22 file86
-rw-r--r-- 1 root root 0 Nov 16 16:22 file87
-rw-r--r-- 1 root root 0 Nov 16 16:22 file88
-rw-r--r-- 1 root root 0 Nov 16 16:22 file89
-rw-r--r-- 1 root root 0 Nov 16 16:22 file9
-rw-r--r-- 1 root root 0 Nov 16 16:22 file90
-rw-r--r-- 1 root root 0 Nov 16 16:22 file91
-rw-r--r-- 1 root root 0 Nov 16 16:22 file92
-rw-r--r-- 1 root root 0 Nov 16 16:22 file93
-rw-r--r-- 1 root root 0 Nov 16 16:22 file94
-rw-r--r-- 1 root root 0 Nov 16 16:22 file95
-rw-r--r-- 1 root root 0 Nov 16 16:22 file96
-rw-r--r-- 1 root root 0 Nov 16 16:22 file97
-rw-r--r-- 1 root root 0 Nov 16 16:22 file98
-rw-r--r-- 1 root root 0 Nov 16 16:22 file99
[root@gfvm3 tierd_volume]# /root/brick_content.sh tiervol
Bricks Present:
/opt/volume_test/tier_vol/b5_2
/opt/volume_test/tier_vol/b5_1
/opt/volume_test/tier_vol/b4_2
/opt/volume_test/tier_vol/b4_1
/opt/volume_test/tier_vol/b1_1
/opt/volume_test/tier_vol/b1_2
/opt/volume_test/tier_vol/b2_1
/opt/volume_test/tier_vol/b2_2
/opt/volume_test/tier_vol/b3_1
/opt/volume_test/tier_vol/b3_2

Contents of Bricks:
/opt/volume_test/tier_vol/b1_1:
total 140
---------T 2 root root 0 Nov 16 16:22 file14
---------T 2 root root 0 Nov 16 16:22 file15
---------T 2 root root 0 Nov 16 16:22 file20
---------T 2 root root 0 Nov 16 16:22 file21
---------T 2 root root 0 Nov 16 16:22 file22
---------T 2 root root 0 Nov 16 16:22 file23
---------T 2 root root 0 Nov 16 16:22 file26
---------T 2 root root 0 Nov 16 16:22 file30
---------T 2 root root 0 Nov 16 16:22 file34
---------T 2 root root 0 Nov 16 16:22 file35
---------T 2 root root 0 Nov 16 16:22 file39
---------T 2 root root 0 Nov 16 16:22 file45
---------T 2 root root 0 Nov 16 16:22 file5
---------T 2 root root 0 Nov 16 16:22 file51
---------T 2 root root 0 Nov 16 16:22 file52
---------T 2 root root 0 Nov 16 16:22 file6
---------T 2 root root 0 Nov 16 16:22 file60
---------T 2 root root 0 Nov 16 16:22 file61
---------T 2 root root 0 Nov 16 16:22 file65
---------T 2 root root 0 Nov 16 16:22 file67
---------T 2 root root 0 Nov 16 16:22 file68
---------T 2 root root 0 Nov 16 16:22 file70
---------T 2 root root 0 Nov 16 16:22 file72
---------T 2 root root 0 Nov 16 16:22 file76
---------T 2 root root 0 Nov 16 16:22 file78
---------T 2 root root 0 Nov 16 16:22 file8
---------T 2 root root 0 Nov 16 16:22 file81
---------T 2 root root 0 Nov 16 16:22 file82
---------T 2 root root 0 Nov 16 16:22 file83
---------T 2 root root 0 Nov 16 16:22 file89
---------T 2 root root 0 Nov 16 16:22 file92
---------T 2 root root 0 Nov 16 16:22 file94
---------T 2 root root 0 Nov 16 16:22 file95
---------T 2 root root 0 Nov 16 16:22 file96
---------T 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b1_2:
total 140
---------T 2 root root 0 Nov 16 16:22 file14
---------T 2 root root 0 Nov 16 16:22 file15
---------T 2 root root 0 Nov 16 16:22 file20
---------T 2 root root 0 Nov 16 16:22 file21
---------T 2 root root 0 Nov 16 16:22 file22
---------T 2 root root 0 Nov 16 16:22 file23
---------T 2 root root 0 Nov 16 16:22 file26
---------T 2 root root 0 Nov 16 16:22 file30
---------T 2 root root 0 Nov 16 16:22 file34
---------T 2 root root 0 Nov 16 16:22 file35
---------T 2 root root 0 Nov 16 16:22 file39
---------T 2 root root 0 Nov 16 16:22 file45
---------T 2 root root 0 Nov 16 16:22 file5
---------T 2 root root 0 Nov 16 16:22 file51
---------T 2 root root 0 Nov 16 16:22 file52
---------T 2 root root 0 Nov 16 16:22 file6
---------T 2 root root 0 Nov 16 16:22 file60
---------T 2 root root 0 Nov 16 16:22 file61
---------T 2 root root 0 Nov 16 16:22 file65
---------T 2 root root 0 Nov 16 16:22 file67
---------T 2 root root 0 Nov 16 16:22 file68
---------T 2 root root 0 Nov 16 16:22 file70
---------T 2 root root 0 Nov 16 16:22 file72
---------T 2 root root 0 Nov 16 16:22 file76
---------T 2 root root 0 Nov 16 16:22 file78
---------T 2 root root 0 Nov 16 16:22 file8
---------T 2 root root 0 Nov 16 16:22 file81
---------T 2 root root 0 Nov 16 16:22 file82
---------T 2 root root 0 Nov 16 16:22 file83
---------T 2 root root 0 Nov 16 16:22 file89
---------T 2 root root 0 Nov 16 16:22 file92
---------T 2 root root 0 Nov 16 16:22 file94
---------T 2 root root 0 Nov 16 16:22 file95
---------T 2 root root 0 Nov 16 16:22 file96
---------T 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b2_1:
total 104
---------T 2 root root 0 Nov 16 16:22 file10
---------T 2 root root 0 Nov 16 16:22 file100
---------T 2 root root 0 Nov 16 16:22 file12
---------T 2 root root 0 Nov 16 16:22 file13
---------T 2 root root 0 Nov 16 16:22 file18
---------T 2 root root 0 Nov 16 16:22 file28
---------T 2 root root 0 Nov 16 16:22 file3
---------T 2 root root 0 Nov 16 16:22 file32
---------T 2 root root 0 Nov 16 16:22 file37
---------T 2 root root 0 Nov 16 16:22 file4
---------T 2 root root 0 Nov 16 16:22 file40
---------T 2 root root 0 Nov 16 16:22 file43
---------T 2 root root 0 Nov 16 16:22 file46
---------T 2 root root 0 Nov 16 16:22 file47
---------T 2 root root 0 Nov 16 16:22 file48
---------T 2 root root 0 Nov 16 16:22 file50
---------T 2 root root 0 Nov 16 16:22 file54
---------T 2 root root 0 Nov 16 16:22 file55
---------T 2 root root 0 Nov 16 16:22 file58
---------T 2 root root 0 Nov 16 16:22 file66
---------T 2 root root 0 Nov 16 16:22 file7
---------T 2 root root 0 Nov 16 16:22 file71
---------T 2 root root 0 Nov 16 16:22 file75
---------T 2 root root 0 Nov 16 16:22 file86
---------T 2 root root 0 Nov 16 16:22 file9
---------T 2 root root 0 Nov 16 16:22 file91

/opt/volume_test/tier_vol/b2_2:
total 104
---------T 2 root root 0 Nov 16 16:22 file10
---------T 2 root root 0 Nov 16 16:22 file100
---------T 2 root root 0 Nov 16 16:22 file12
---------T 2 root root 0 Nov 16 16:22 file13
---------T 2 root root 0 Nov 16 16:22 file18
---------T 2 root root 0 Nov 16 16:22 file28
---------T 2 root root 0 Nov 16 16:22 file3
---------T 2 root root 0 Nov 16 16:22 file32
---------T 2 root root 0 Nov 16 16:22 file37
---------T 2 root root 0 Nov 16 16:22 file4
---------T 2 root root 0 Nov 16 16:22 file40
---------T 2 root root 0 Nov 16 16:22 file43
---------T 2 root root 0 Nov 16 16:22 file46
---------T 2 root root 0 Nov 16 16:22 file47
---------T 2 root root 0 Nov 16 16:22 file48
---------T 2 root root 0 Nov 16 16:22 file50
---------T 2 root root 0 Nov 16 16:22 file54
---------T 2 root root 0 Nov 16 16:22 file55
---------T 2 root root 0 Nov 16 16:22 file58
---------T 2 root root 0 Nov 16 16:22 file66
---------T 2 root root 0 Nov 16 16:22 file7
---------T 2 root root 0 Nov 16 16:22 file71
---------T 2 root root 0 Nov 16 16:22 file75
---------T 2 root root 0 Nov 16 16:22 file86
---------T 2 root root 0 Nov 16 16:22 file9
---------T 2 root root 0 Nov 16 16:22 file91

/opt/volume_test/tier_vol/b3_1:
total 156
---------T 2 root root 0 Nov 16 16:22 file1
---------T 2 root root 0 Nov 16 16:22 file11
---------T 2 root root 0 Nov 16 16:22 file16
---------T 2 root root 0 Nov 16 16:22 file17
---------T 2 root root 0 Nov 16 16:22 file19
---------T 2 root root 0 Nov 16 16:22 file2
---------T 2 root root 0 Nov 16 16:22 file24
---------T 2 root root 0 Nov 16 16:22 file25
---------T 2 root root 0 Nov 16 16:22 file27
---------T 2 root root 0 Nov 16 16:22 file29
---------T 2 root root 0 Nov 16 16:22 file31
---------T 2 root root 0 Nov 16 16:22 file33
---------T 2 root root 0 Nov 16 16:22 file36
---------T 2 root root 0 Nov 16 16:22 file38
---------T 2 root root 0 Nov 16 16:22 file41
---------T 2 root root 0 Nov 16 16:22 file42
---------T 2 root root 0 Nov 16 16:22 file44
---------T 2 root root 0 Nov 16 16:22 file49
---------T 2 root root 0 Nov 16 16:22 file53
---------T 2 root root 0 Nov 16 16:22 file56
---------T 2 root root 0 Nov 16 16:22 file57
---------T 2 root root 0 Nov 16 16:22 file59
---------T 2 root root 0 Nov 16 16:22 file62
---------T 2 root root 0 Nov 16 16:22 file63
---------T 2 root root 0 Nov 16 16:22 file64
---------T 2 root root 0 Nov 16 16:22 file69
---------T 2 root root 0 Nov 16 16:22 file73
---------T 2 root root 0 Nov 16 16:22 file74
---------T 2 root root 0 Nov 16 16:22 file77
---------T 2 root root 0 Nov 16 16:22 file79
---------T 2 root root 0 Nov 16 16:22 file80
---------T 2 root root 0 Nov 16 16:22 file84
---------T 2 root root 0 Nov 16 16:22 file85
---------T 2 root root 0 Nov 16 16:22 file87
---------T 2 root root 0 Nov 16 16:22 file88
---------T 2 root root 0 Nov 16 16:22 file90
---------T 2 root root 0 Nov 16 16:22 file93
---------T 2 root root 0 Nov 16 16:22 file97
---------T 2 root root 0 Nov 16 16:22 file98

/opt/volume_test/tier_vol/b3_2:
total 156
---------T 2 root root 0 Nov 16 16:22 file1
---------T 2 root root 0 Nov 16 16:22 file11
---------T 2 root root 0 Nov 16 16:22 file16
---------T 2 root root 0 Nov 16 16:22 file17
---------T 2 root root 0 Nov 16 16:22 file19
---------T 2 root root 0 Nov 16 16:22 file2
---------T 2 root root 0 Nov 16 16:22 file24
---------T 2 root root 0 Nov 16 16:22 file25
---------T 2 root root 0 Nov 16 16:22 file27
---------T 2 root root 0 Nov 16 16:22 file29
---------T 2 root root 0 Nov 16 16:22 file31
---------T 2 root root 0 Nov 16 16:22 file33
---------T 2 root root 0 Nov 16 16:22 file36
---------T 2 root root 0 Nov 16 16:22 file38
---------T 2 root root 0 Nov 16 16:22 file41
---------T 2 root root 0 Nov 16 16:22 file42
---------T 2 root root 0 Nov 16 16:22 file44
---------T 2 root root 0 Nov 16 16:22 file49
---------T 2 root root 0 Nov 16 16:22 file53
---------T 2 root root 0 Nov 16 16:22 file56
---------T 2 root root 0 Nov 16 16:22 file57
---------T 2 root root 0 Nov 16 16:22 file59
---------T 2 root root 0 Nov 16 16:22 file62
---------T 2 root root 0 Nov 16 16:22 file63
---------T 2 root root 0 Nov 16 16:22 file64
---------T 2 root root 0 Nov 16 16:22 file69
---------T 2 root root 0 Nov 16 16:22 file73
---------T 2 root root 0 Nov 16 16:22 file74
---------T 2 root root 0 Nov 16 16:22 file77
---------T 2 root root 0 Nov 16 16:22 file79
---------T 2 root root 0 Nov 16 16:22 file80
---------T 2 root root 0 Nov 16 16:22 file84
---------T 2 root root 0 Nov 16 16:22 file85
---------T 2 root root 0 Nov 16 16:22 file87
---------T 2 root root 0 Nov 16 16:22 file88
---------T 2 root root 0 Nov 16 16:22 file90
---------T 2 root root 0 Nov 16 16:22 file93
---------T 2 root root 0 Nov 16 16:22 file97
---------T 2 root root 0 Nov 16 16:22 file98

/opt/volume_test/tier_vol/b4_1:
total 216
-rw-r--r-- 2 root root 0 Nov 16 16:22 file1
-rw-r--r-- 2 root root 0 Nov 16 16:22 file14
-rw-r--r-- 2 root root 0 Nov 16 16:22 file15
-rw-r--r-- 2 root root 0 Nov 16 16:22 file2
-rw-r--r-- 2 root root 0 Nov 16 16:22 file20
-rw-r--r-- 2 root root 0 Nov 16 16:22 file21
-rw-r--r-- 2 root root 0 Nov 16 16:22 file22
-rw-r--r-- 2 root root 0 Nov 16 16:22 file23
-rw-r--r-- 2 root root 0 Nov 16 16:22 file26
-rw-r--r-- 2 root root 0 Nov 16 16:22 file27
-rw-r--r-- 2 root root 0 Nov 16 16:22 file29
-rw-r--r-- 2 root root 0 Nov 16 16:22 file30
-rw-r--r-- 2 root root 0 Nov 16 16:22 file33
-rw-r--r-- 2 root root 0 Nov 16 16:22 file34
-rw-r--r-- 2 root root 0 Nov 16 16:22 file35
-rw-r--r-- 2 root root 0 Nov 16 16:22 file36
-rw-r--r-- 2 root root 0 Nov 16 16:22 file39
-rw-r--r-- 2 root root 0 Nov 16 16:22 file45
-rw-r--r-- 2 root root 0 Nov 16 16:22 file49
-rw-r--r-- 2 root root 0 Nov 16 16:22 file5
-rw-r--r-- 2 root root 0 Nov 16 16:22 file51
-rw-r--r-- 2 root root 0 Nov 16 16:22 file52
-rw-r--r-- 2 root root 0 Nov 16 16:22 file56
-rw-r--r-- 2 root root 0 Nov 16 16:22 file57
-rw-r--r-- 2 root root 0 Nov 16 16:22 file59
-rw-r--r-- 2 root root 0 Nov 16 16:22 file6
-rw-r--r-- 2 root root 0 Nov 16 16:22 file60
-rw-r--r-- 2 root root 0 Nov 16 16:22 file61
-rw-r--r-- 2 root root 0 Nov 16 16:22 file64
-rw-r--r-- 2 root root 0 Nov 16 16:22 file65
-rw-r--r-- 2 root root 0 Nov 16 16:22 file67
-rw-r--r-- 2 root root 0 Nov 16 16:22 file68
-rw-r--r-- 2 root root 0 Nov 16 16:22 file70
-rw-r--r-- 2 root root 0 Nov 16 16:22 file72
-rw-r--r-- 2 root root 0 Nov 16 16:22 file74
-rw-r--r-- 2 root root 0 Nov 16 16:22 file76
-rw-r--r-- 2 root root 0 Nov 16 16:22 file77
-rw-r--r-- 2 root root 0 Nov 16 16:22 file78
-rw-r--r-- 2 root root 0 Nov 16 16:22 file79
-rw-r--r-- 2 root root 0 Nov 16 16:22 file8
-rw-r--r-- 2 root root 0 Nov 16 16:22 file81
-rw-r--r-- 2 root root 0 Nov 16 16:22 file82
-rw-r--r-- 2 root root 0 Nov 16 16:22 file83
-rw-r--r-- 2 root root 0 Nov 16 16:22 file84
-rw-r--r-- 2 root root 0 Nov 16 16:22 file85
-rw-r--r-- 2 root root 0 Nov 16 16:22 file88
-rw-r--r-- 2 root root 0 Nov 16 16:22 file89
-rw-r--r-- 2 root root 0 Nov 16 16:22 file90
-rw-r--r-- 2 root root 0 Nov 16 16:22 file92
-rw-r--r-- 2 root root 0 Nov 16 16:22 file94
-rw-r--r-- 2 root root 0 Nov 16 16:22 file95
-rw-r--r-- 2 root root 0 Nov 16 16:22 file96
-rw-r--r-- 2 root root 0 Nov 16 16:22 file98
-rw-r--r-- 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b4_2:
total 216
-rw-r--r-- 2 root root 0 Nov 16 16:22 file1
-rw-r--r-- 2 root root 0 Nov 16 16:22 file14
-rw-r--r-- 2 root root 0 Nov 16 16:22 file15
-rw-r--r-- 2 root root 0 Nov 16 16:22 file2
-rw-r--r-- 2 root root 0 Nov 16 16:22 file20
-rw-r--r-- 2 root root 0 Nov 16 16:22 file21
-rw-r--r-- 2 root root 0 Nov 16 16:22 file22
-rw-r--r-- 2 root root 0 Nov 16 16:22 file23
-rw-r--r-- 2 root root 0 Nov 16 16:22 file26
-rw-r--r-- 2 root root 0 Nov 16 16:22 file27
-rw-r--r-- 2 root root 0 Nov 16 16:22 file29
-rw-r--r-- 2 root root 0 Nov 16 16:22 file30
-rw-r--r-- 2 root root 0 Nov 16 16:22 file33
-rw-r--r-- 2 root root 0 Nov 16 16:22 file34
-rw-r--r-- 2 root root 0 Nov 16 16:22 file35
-rw-r--r-- 2 root root 0 Nov 16 16:22 file36
-rw-r--r-- 2 root root 0 Nov 16 16:22 file39
-rw-r--r-- 2 root root 0 Nov 16 16:22 file45
-rw-r--r-- 2 root root 0 Nov 16 16:22 file49
-rw-r--r-- 2 root root 0 Nov 16 16:22 file5
-rw-r--r-- 2 root root 0 Nov 16 16:22 file51
-rw-r--r-- 2 root root 0 Nov 16 16:22 file52
-rw-r--r-- 2 root root 0 Nov 16 16:22 file56
-rw-r--r-- 2 root root 0 Nov 16 16:22 file57
-rw-r--r-- 2 root root 0 Nov 16 16:22 file59
-rw-r--r-- 2 root root 0 Nov 16 16:22 file6
-rw-r--r-- 2 root root 0 Nov 16 16:22 file60
-rw-r--r-- 2 root root 0 Nov 16 16:22 file61
-rw-r--r-- 2 root root 0 Nov 16 16:22 file64
-rw-r--r-- 2 root root 0 Nov 16 16:22 file65
-rw-r--r-- 2 root root 0 Nov 16 16:22 file67
-rw-r--r-- 2 root root 0 Nov 16 16:22 file68
-rw-r--r-- 2 root root 0 Nov 16 16:22 file70
-rw-r--r-- 2 root root 0 Nov 16 16:22 file72
-rw-r--r-- 2 root root 0 Nov 16 16:22 file74
-rw-r--r-- 2 root root 0 Nov 16 16:22 file76
-rw-r--r-- 2 root root 0 Nov 16 16:22 file77
-rw-r--r-- 2 root root 0 Nov 16 16:22 file78
-rw-r--r-- 2 root root 0 Nov 16 16:22 file79
-rw-r--r-- 2 root root 0 Nov 16 16:22 file8
-rw-r--r-- 2 root root 0 Nov 16 16:22 file81
-rw-r--r-- 2 root root 0 Nov 16 16:22 file82
-rw-r--r-- 2 root root 0 Nov 16 16:22 file83
-rw-r--r-- 2 root root 0 Nov 16 16:22 file84
-rw-r--r-- 2 root root 0 Nov 16 16:22 file85
-rw-r--r-- 2 root root 0 Nov 16 16:22 file88
-rw-r--r-- 2 root root 0 Nov 16 16:22 file89
-rw-r--r-- 2 root root 0 Nov 16 16:22 file90
-rw-r--r-- 2 root root 0 Nov 16 16:22 file92
-rw-r--r-- 2 root root 0 Nov 16 16:22 file94
-rw-r--r-- 2 root root 0 Nov 16 16:22 file95
-rw-r--r-- 2 root root 0 Nov 16 16:22 file96
-rw-r--r-- 2 root root 0 Nov 16 16:22 file98
-rw-r--r-- 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b5_1:
total 184
-rw-r--r-- 2 root root 0 Nov 16 16:22 file10
-rw-r--r-- 2 root root 0 Nov 16 16:22 file100
-rw-r--r-- 2 root root 0 Nov 16 16:22 file11
-rw-r--r-- 2 root root 0 Nov 16 16:22 file12
-rw-r--r-- 2 root root 0 Nov 16 16:22 file13
-rw-r--r-- 2 root root 0 Nov 16 16:22 file16
-rw-r--r-- 2 root root 0 Nov 16 16:22 file17
-rw-r--r-- 2 root root 0 Nov 16 16:22 file18
-rw-r--r-- 2 root root 0 Nov 16 16:22 file19
-rw-r--r-- 2 root root 0 Nov 16 16:22 file24
-rw-r--r-- 2 root root 0 Nov 16 16:22 file25
-rw-r--r-- 2 root root 0 Nov 16 16:22 file28
-rw-r--r-- 2 root root 0 Nov 16 16:22 file3
-rw-r--r-- 2 root root 0 Nov 16 16:22 file31
-rw-r--r-- 2 root root 0 Nov 16 16:22 file32
-rw-r--r-- 2 root root 0 Nov 16 16:22 file37
-rw-r--r-- 2 root root 0 Nov 16 16:22 file38
-rw-r--r-- 2 root root 0 Nov 16 16:22 file4
-rw-r--r-- 2 root root 0 Nov 16 16:22 file40
-rw-r--r-- 2 root root 0 Nov 16 16:22 file41
-rw-r--r-- 2 root root 0 Nov 16 16:22 file42
-rw-r--r-- 2 root root 0 Nov 16 16:22 file43
-rw-r--r-- 2 root root 0 Nov 16 16:22 file44
-rw-r--r-- 2 root root 0 Nov 16 16:22 file46
-rw-r--r-- 2 root root 0 Nov 16 16:22 file47
-rw-r--r-- 2 root root 0 Nov 16 16:22 file48
-rw-r--r-- 2 root root 0 Nov 16 16:22 file50
-rw-r--r-- 2 root root 0 Nov 16 16:22 file53
-rw-r--r-- 2 root root 0 Nov 16 16:22 file54
-rw-r--r-- 2 root root 0 Nov 16 16:22 file55
-rw-r--r-- 2 root root 0 Nov 16 16:22 file58
-rw-r--r-- 2 root root 0 Nov 16 16:22 file62
-rw-r--r-- 2 root root 0 Nov 16 16:22 file63
-rw-r--r-- 2 root root 0 Nov 16 16:22 file66
-rw-r--r-- 2 root root 0 Nov 16 16:22 file69
-rw-r--r-- 2 root root 0 Nov 16 16:22 file7
-rw-r--r-- 2 root root 0 Nov 16 16:22 file71
-rw-r--r-- 2 root root 0 Nov 16 16:22 file73
-rw-r--r-- 2 root root 0 Nov 16 16:22 file75
-rw-r--r-- 2 root root 0 Nov 16 16:22 file80
-rw-r--r-- 2 root root 0 Nov 16 16:22 file86
-rw-r--r-- 2 root root 0 Nov 16 16:22 file87
-rw-r--r-- 2 root root 0 Nov 16 16:22 file9
-rw-r--r-- 2 root root 0 Nov 16 16:22 file91
-rw-r--r-- 2 root root 0 Nov 16 16:22 file93
-rw-r--r-- 2 root root 0 Nov 16 16:22 file97

/opt/volume_test/tier_vol/b5_2:
total 184
-rw-r--r-- 2 root root 0 Nov 16 16:22 file10
-rw-r--r-- 2 root root 0 Nov 16 16:22 file100
-rw-r--r-- 2 root root 0 Nov 16 16:22 file11
-rw-r--r-- 2 root root 0 Nov 16 16:22 file12
-rw-r--r-- 2 root root 0 Nov 16 16:22 file13
-rw-r--r-- 2 root root 0 Nov 16 16:22 file16
-rw-r--r-- 2 root root 0 Nov 16 16:22 file17
-rw-r--r-- 2 root root 0 Nov 16 16:22 file18
-rw-r--r-- 2 root root 0 Nov 16 16:22 file19
-rw-r--r-- 2 root root 0 Nov 16 16:22 file24
-rw-r--r-- 2 root root 0 Nov 16 16:22 file25
-rw-r--r-- 2 root root 0 Nov 16 16:22 file28
-rw-r--r-- 2 root root 0 Nov 16 16:22 file3
-rw-r--r-- 2 root root 0 Nov 16 16:22 file31
-rw-r--r-- 2 root root 0 Nov 16 16:22 file32
-rw-r--r-- 2 root root 0 Nov 16 16:22 file37
-rw-r--r-- 2 root root 0 Nov 16 16:22 file38
-rw-r--r-- 2 root root 0 Nov 16 16:22 file4
-rw-r--r-- 2 root root 0 Nov 16 16:22 file40
-rw-r--r-- 2 root root 0 Nov 16 16:22 file41
-rw-r--r-- 2 root root 0 Nov 16 16:22 file42
-rw-r--r-- 2 root root 0 Nov 16 16:22 file43
-rw-r--r-- 2 root root 0 Nov 16 16:22 file44
-rw-r--r-- 2 root root 0 Nov 16 16:22 file46
-rw-r--r-- 2 root root 0 Nov 16 16:22 file47
-rw-r--r-- 2 root root 0 Nov 16 16:22 file48
-rw-r--r-- 2 root root 0 Nov 16 16:22 file50
-rw-r--r-- 2 root root 0 Nov 16 16:22 file53
-rw-r--r-- 2 root root 0 Nov 16 16:22 file54
-rw-r--r-- 2 root root 0 Nov 16 16:22 file55
-rw-r--r-- 2 root root 0 Nov 16 16:22 file58
-rw-r--r-- 2 root root 0 Nov 16 16:22 file62
-rw-r--r-- 2 root root 0 Nov 16 16:22 file63
-rw-r--r-- 2 root root 0 Nov 16 16:22 file66
-rw-r--r-- 2 root root 0 Nov 16 16:22 file69
-rw-r--r-- 2 root root 0 Nov 16 16:22 file7
-rw-r--r-- 2 root root 0 Nov 16 16:22 file71
-rw-r--r-- 2 root root 0 Nov 16 16:22 file73
-rw-r--r-- 2 root root 0 Nov 16 16:22 file75
-rw-r--r-- 2 root root 0 Nov 16 16:22 file80
-rw-r--r-- 2 root root 0 Nov 16 16:22 file86
-rw-r--r-- 2 root root 0 Nov 16 16:22 file87
-rw-r--r-- 2 root root 0 Nov 16 16:22 file9
-rw-r--r-- 2 root root 0 Nov 16 16:22 file91
-rw-r--r-- 2 root root 0 Nov 16 16:22 file93
-rw-r--r-- 2 root root 0 Nov 16 16:22 file97
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]# gluster volume set tiervol cluster.tier-demote-frequency  5
volume set: success
[root@gfvm3 tierd_volume]# sleep 5; /root/brick_content.sh tiervol
Bricks Present:
/opt/volume_test/tier_vol/b5_2
/opt/volume_test/tier_vol/b5_1
/opt/volume_test/tier_vol/b4_2
/opt/volume_test/tier_vol/b4_1
/opt/volume_test/tier_vol/b1_1
/opt/volume_test/tier_vol/b1_2
/opt/volume_test/tier_vol/b2_1
/opt/volume_test/tier_vol/b2_2
/opt/volume_test/tier_vol/b3_1
/opt/volume_test/tier_vol/b3_2

Contents of Bricks:
/opt/volume_test/tier_vol/b1_1:
total 140
-rw-r--r-- 2 root root 0 Nov 16 16:22 file14
-rw-r--r-- 2 root root 0 Nov 16 16:22 file15
-rw-r--r-- 2 root root 0 Nov 16 16:22 file20
-rw-r--r-- 2 root root 0 Nov 16 16:22 file21
-rw-r--r-- 2 root root 0 Nov 16 16:22 file22
-rw-r--r-- 2 root root 0 Nov 16 16:22 file23
-rw-r--r-- 2 root root 0 Nov 16 16:22 file26
-rw-r--r-- 2 root root 0 Nov 16 16:22 file30
-rw-r--r-- 2 root root 0 Nov 16 16:22 file34
-rw-r--r-- 2 root root 0 Nov 16 16:22 file35
-rw-r--r-- 2 root root 0 Nov 16 16:22 file39
-rw-r--r-- 2 root root 0 Nov 16 16:22 file45
-rw-r--r-- 2 root root 0 Nov 16 16:22 file5
-rw-r--r-- 2 root root 0 Nov 16 16:22 file51
-rw-r--r-- 2 root root 0 Nov 16 16:22 file52
-rw-r--r-- 2 root root 0 Nov 16 16:22 file6
-rw-r--r-- 2 root root 0 Nov 16 16:22 file60
-rw-r--r-- 2 root root 0 Nov 16 16:22 file61
-rw-r--r-- 2 root root 0 Nov 16 16:22 file65
-rw-r--r-- 2 root root 0 Nov 16 16:22 file67
-rw-r--r-- 2 root root 0 Nov 16 16:22 file68
-rw-r--r-- 2 root root 0 Nov 16 16:22 file70
-rw-r--r-- 2 root root 0 Nov 16 16:22 file72
-rw-r--r-- 2 root root 0 Nov 16 16:22 file76
-rw-r--r-- 2 root root 0 Nov 16 16:22 file78
-rw-r--r-- 2 root root 0 Nov 16 16:22 file8
-rw-r--r-- 2 root root 0 Nov 16 16:22 file81
-rw-r--r-- 2 root root 0 Nov 16 16:22 file82
-rw-r--r-- 2 root root 0 Nov 16 16:22 file83
-rw-r--r-- 2 root root 0 Nov 16 16:22 file89
-rw-r--r-- 2 root root 0 Nov 16 16:22 file92
-rw-r--r-- 2 root root 0 Nov 16 16:22 file94
-rw-r--r-- 2 root root 0 Nov 16 16:22 file95
-rw-r--r-- 2 root root 0 Nov 16 16:22 file96
-rw-r--r-- 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b1_2:
total 140
-rw-r--r-- 2 root root 0 Nov 16 16:22 file14
-rw-r--r-- 2 root root 0 Nov 16 16:22 file15
-rw-r--r-- 2 root root 0 Nov 16 16:22 file20
-rw-r--r-- 2 root root 0 Nov 16 16:22 file21
-rw-r--r-- 2 root root 0 Nov 16 16:22 file22
-rw-r--r-- 2 root root 0 Nov 16 16:22 file23
-rw-r--r-- 2 root root 0 Nov 16 16:22 file26
-rw-r--r-- 2 root root 0 Nov 16 16:22 file30
-rw-r--r-- 2 root root 0 Nov 16 16:22 file34
-rw-r--r-- 2 root root 0 Nov 16 16:22 file35
-rw-r--r-- 2 root root 0 Nov 16 16:22 file39
-rw-r--r-- 2 root root 0 Nov 16 16:22 file45
-rw-r--r-- 2 root root 0 Nov 16 16:22 file5
-rw-r--r-- 2 root root 0 Nov 16 16:22 file51
-rw-r--r-- 2 root root 0 Nov 16 16:22 file52
-rw-r--r-- 2 root root 0 Nov 16 16:22 file6
-rw-r--r-- 2 root root 0 Nov 16 16:22 file60
-rw-r--r-- 2 root root 0 Nov 16 16:22 file61
-rw-r--r-- 2 root root 0 Nov 16 16:22 file65
-rw-r--r-- 2 root root 0 Nov 16 16:22 file67
-rw-r--r-- 2 root root 0 Nov 16 16:22 file68
-rw-r--r-- 2 root root 0 Nov 16 16:22 file70
-rw-r--r-- 2 root root 0 Nov 16 16:22 file72
-rw-r--r-- 2 root root 0 Nov 16 16:22 file76
-rw-r--r-- 2 root root 0 Nov 16 16:22 file78
-rw-r--r-- 2 root root 0 Nov 16 16:22 file8
-rw-r--r-- 2 root root 0 Nov 16 16:22 file81
-rw-r--r-- 2 root root 0 Nov 16 16:22 file82
-rw-r--r-- 2 root root 0 Nov 16 16:22 file83
-rw-r--r-- 2 root root 0 Nov 16 16:22 file89
-rw-r--r-- 2 root root 0 Nov 16 16:22 file92
-rw-r--r-- 2 root root 0 Nov 16 16:22 file94
-rw-r--r-- 2 root root 0 Nov 16 16:22 file95
-rw-r--r-- 2 root root 0 Nov 16 16:22 file96
-rw-r--r-- 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b2_1:
total 104
-rw-r--r-- 2 root root 0 Nov 16 16:22 file10
-rw-r--r-- 2 root root 0 Nov 16 16:22 file100
-rw-r--r-- 2 root root 0 Nov 16 16:22 file12
-rw-r--r-- 2 root root 0 Nov 16 16:22 file13
-rw-r--r-- 2 root root 0 Nov 16 16:22 file18
-rw-r--r-- 2 root root 0 Nov 16 16:22 file28
-rw-r--r-- 2 root root 0 Nov 16 16:22 file3
-rw-r--r-- 2 root root 0 Nov 16 16:22 file32
-rw-r--r-- 2 root root 0 Nov 16 16:22 file37
-rw-r--r-- 2 root root 0 Nov 16 16:22 file4
-rw-r--r-- 2 root root 0 Nov 16 16:22 file40
-rw-r--r-- 2 root root 0 Nov 16 16:22 file43
-rw-r--r-- 2 root root 0 Nov 16 16:22 file46
-rw-r--r-- 2 root root 0 Nov 16 16:22 file47
-rw-r--r-- 2 root root 0 Nov 16 16:22 file48
-rw-r--r-- 2 root root 0 Nov 16 16:22 file50
-rw-r--r-- 2 root root 0 Nov 16 16:22 file54
-rw-r--r-- 2 root root 0 Nov 16 16:22 file55
-rw-r--r-- 2 root root 0 Nov 16 16:22 file58
-rw-r--r-- 2 root root 0 Nov 16 16:22 file66
-rw-r--r-- 2 root root 0 Nov 16 16:22 file7
-rw-r--r-- 2 root root 0 Nov 16 16:22 file71
-rw-r--r-- 2 root root 0 Nov 16 16:22 file75
-rw-r--r-- 2 root root 0 Nov 16 16:22 file86
-rw-r--r-- 2 root root 0 Nov 16 16:22 file9
-rw-r--r-- 2 root root 0 Nov 16 16:22 file91

/opt/volume_test/tier_vol/b2_2:
total 104
-rw-r--r-- 2 root root 0 Nov 16 16:22 file10
-rw-r--r-- 2 root root 0 Nov 16 16:22 file100
-rw-r--r-- 2 root root 0 Nov 16 16:22 file12
-rw-r--r-- 2 root root 0 Nov 16 16:22 file13
-rw-r--r-- 2 root root 0 Nov 16 16:22 file18
-rw-r--r-- 2 root root 0 Nov 16 16:22 file28
-rw-r--r-- 2 root root 0 Nov 16 16:22 file3
-rw-r--r-- 2 root root 0 Nov 16 16:22 file32
-rw-r--r-- 2 root root 0 Nov 16 16:22 file37
-rw-r--r-- 2 root root 0 Nov 16 16:22 file4
-rw-r--r-- 2 root root 0 Nov 16 16:22 file40
-rw-r--r-- 2 root root 0 Nov 16 16:22 file43
-rw-r--r-- 2 root root 0 Nov 16 16:22 file46
-rw-r--r-- 2 root root 0 Nov 16 16:22 file47
-rw-r--r-- 2 root root 0 Nov 16 16:22 file48
-rw-r--r-- 2 root root 0 Nov 16 16:22 file50
-rw-r--r-- 2 root root 0 Nov 16 16:22 file54
-rw-r--r-- 2 root root 0 Nov 16 16:22 file55
-rw-r--r-- 2 root root 0 Nov 16 16:22 file58
-rw-r--r-- 2 root root 0 Nov 16 16:22 file66
-rw-r--r-- 2 root root 0 Nov 16 16:22 file7
-rw-r--r-- 2 root root 0 Nov 16 16:22 file71
-rw-r--r-- 2 root root 0 Nov 16 16:22 file75
-rw-r--r-- 2 root root 0 Nov 16 16:22 file86
-rw-r--r-- 2 root root 0 Nov 16 16:22 file9
-rw-r--r-- 2 root root 0 Nov 16 16:22 file91

/opt/volume_test/tier_vol/b3_1:
total 156
-rw-r--r-- 2 root root 0 Nov 16 16:22 file1
-rw-r--r-- 2 root root 0 Nov 16 16:22 file11
-rw-r--r-- 2 root root 0 Nov 16 16:22 file16
-rw-r--r-- 2 root root 0 Nov 16 16:22 file17
-rw-r--r-- 2 root root 0 Nov 16 16:22 file19
-rw-r--r-- 2 root root 0 Nov 16 16:22 file2
-rw-r--r-- 2 root root 0 Nov 16 16:22 file24
-rw-r--r-- 2 root root 0 Nov 16 16:22 file25
-rw-r--r-- 2 root root 0 Nov 16 16:22 file27
-rw-r--r-- 2 root root 0 Nov 16 16:22 file29
-rw-r--r-- 2 root root 0 Nov 16 16:22 file31
-rw-r--r-- 2 root root 0 Nov 16 16:22 file33
-rw-r--r-- 2 root root 0 Nov 16 16:22 file36
-rw-r--r-- 2 root root 0 Nov 16 16:22 file38
-rw-r--r-- 2 root root 0 Nov 16 16:22 file41
-rw-r--r-- 2 root root 0 Nov 16 16:22 file42
-rw-r--r-- 2 root root 0 Nov 16 16:22 file44
-rw-r--r-- 2 root root 0 Nov 16 16:22 file49
-rw-r--r-- 2 root root 0 Nov 16 16:22 file53
-rw-r--r-- 2 root root 0 Nov 16 16:22 file56
-rw-r--r-- 2 root root 0 Nov 16 16:22 file57
-rw-r--r-- 2 root root 0 Nov 16 16:22 file59
-rw-r--r-- 2 root root 0 Nov 16 16:22 file62
-rw-r--r-- 2 root root 0 Nov 16 16:22 file63
-rw-r--r-- 2 root root 0 Nov 16 16:22 file64
-rw-r--r-- 2 root root 0 Nov 16 16:22 file69
-rw-r--r-- 2 root root 0 Nov 16 16:22 file73
-rw-r--r-- 2 root root 0 Nov 16 16:22 file74
-rw-r--r-- 2 root root 0 Nov 16 16:22 file77
-rw-r--r-- 2 root root 0 Nov 16 16:22 file79
-rw-r--r-- 2 root root 0 Nov 16 16:22 file80
-rw-r--r-- 2 root root 0 Nov 16 16:22 file84
-rw-r--r-- 2 root root 0 Nov 16 16:22 file85
-rw-r--r-- 2 root root 0 Nov 16 16:22 file87
-rw-r--r-- 2 root root 0 Nov 16 16:22 file88
-rw-r--r-- 2 root root 0 Nov 16 16:22 file90
-rw-r--r-- 2 root root 0 Nov 16 16:22 file93
-rw-r--r-- 2 root root 0 Nov 16 16:22 file97
-rw-r--r-- 2 root root 0 Nov 16 16:22 file98

/opt/volume_test/tier_vol/b3_2:
total 156
-rw-r--r-- 2 root root 0 Nov 16 16:22 file1
-rw-r--r-- 2 root root 0 Nov 16 16:22 file11
-rw-r--r-- 2 root root 0 Nov 16 16:22 file16
-rw-r--r-- 2 root root 0 Nov 16 16:22 file17
-rw-r--r-- 2 root root 0 Nov 16 16:22 file19
-rw-r--r-- 2 root root 0 Nov 16 16:22 file2
-rw-r--r-- 2 root root 0 Nov 16 16:22 file24
-rw-r--r-- 2 root root 0 Nov 16 16:22 file25
-rw-r--r-- 2 root root 0 Nov 16 16:22 file27
-rw-r--r-- 2 root root 0 Nov 16 16:22 file29
-rw-r--r-- 2 root root 0 Nov 16 16:22 file31
-rw-r--r-- 2 root root 0 Nov 16 16:22 file33
-rw-r--r-- 2 root root 0 Nov 16 16:22 file36
-rw-r--r-- 2 root root 0 Nov 16 16:22 file38
-rw-r--r-- 2 root root 0 Nov 16 16:22 file41
-rw-r--r-- 2 root root 0 Nov 16 16:22 file42
-rw-r--r-- 2 root root 0 Nov 16 16:22 file44
-rw-r--r-- 2 root root 0 Nov 16 16:22 file49
-rw-r--r-- 2 root root 0 Nov 16 16:22 file53
-rw-r--r-- 2 root root 0 Nov 16 16:22 file56
-rw-r--r-- 2 root root 0 Nov 16 16:22 file57
-rw-r--r-- 2 root root 0 Nov 16 16:22 file59
-rw-r--r-- 2 root root 0 Nov 16 16:22 file62
-rw-r--r-- 2 root root 0 Nov 16 16:22 file63
-rw-r--r-- 2 root root 0 Nov 16 16:22 file64
-rw-r--r-- 2 root root 0 Nov 16 16:22 file69
-rw-r--r-- 2 root root 0 Nov 16 16:22 file73
-rw-r--r-- 2 root root 0 Nov 16 16:22 file74
-rw-r--r-- 2 root root 0 Nov 16 16:22 file77
-rw-r--r-- 2 root root 0 Nov 16 16:22 file79
-rw-r--r-- 2 root root 0 Nov 16 16:22 file80
-rw-r--r-- 2 root root 0 Nov 16 16:22 file84
-rw-r--r-- 2 root root 0 Nov 16 16:22 file85
-rw-r--r-- 2 root root 0 Nov 16 16:22 file87
-rw-r--r-- 2 root root 0 Nov 16 16:22 file88
-rw-r--r-- 2 root root 0 Nov 16 16:22 file90
-rw-r--r-- 2 root root 0 Nov 16 16:22 file93
-rw-r--r-- 2 root root 0 Nov 16 16:22 file97
-rw-r--r-- 2 root root 0 Nov 16 16:22 file98

/opt/volume_test/tier_vol/b4_1:
total 0

/opt/volume_test/tier_vol/b4_2:
total 0

/opt/volume_test/tier_vol/b5_1:
total 0

/opt/volume_test/tier_vol/b5_2:
total 0
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]# ls -l | wc -l
101
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]# ls -l /mnt/slave | wc -l
101
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
=================================================================
PART 2:
=================================================================
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]# gluster volume get tiervol cluster.tier-demote-frequency
Option                                  Value                                   
------                                  -----                                   
cluster.tier-demote-frequency           5                                       
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]# gluster volume set tiervol cluster.tier-demote-frequency 50
volume set: success
[root@gfvm3 tierd_volume]# gluster volume get tiervol cluster.tier-promote-frequency
Option                                  Value                                   
------                                  -----                                   
cluster.tier-promote-frequency          120                                     
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]#
[root@gfvm3 tierd_volume]# gluster volume set tiervol cluster.tier-promote-frequency 10
volume set: success
[root@gfvm3 tierd_volume]# sleep 10; /root/brick_content.sh tiervol
Bricks Present:
/opt/volume_test/tier_vol/b5_2
/opt/volume_test/tier_vol/b5_1
/opt/volume_test/tier_vol/b4_2
/opt/volume_test/tier_vol/b4_1
/opt/volume_test/tier_vol/b1_1
/opt/volume_test/tier_vol/b1_2
/opt/volume_test/tier_vol/b2_1
/opt/volume_test/tier_vol/b2_2
/opt/volume_test/tier_vol/b3_1
/opt/volume_test/tier_vol/b3_2

Contents of Bricks:
/opt/volume_test/tier_vol/b1_1:
total 140
-rw-r--r-- 2 root root 0 Nov 16 16:22 file14
-rw-r--r-- 2 root root 0 Nov 16 16:22 file15
-rw-r--r-- 2 root root 0 Nov 16 16:22 file20
-rw-r--r-- 2 root root 0 Nov 16 16:22 file21
-rw-r--r-- 2 root root 0 Nov 16 16:22 file22
-rw-r--r-- 2 root root 0 Nov 16 16:22 file23
-rw-r--r-- 2 root root 0 Nov 16 16:22 file26
-rw-r--r-- 2 root root 0 Nov 16 16:22 file30
-rw-r--r-- 2 root root 0 Nov 16 16:22 file34
-rw-r--r-- 2 root root 0 Nov 16 16:22 file35
-rw-r--r-- 2 root root 0 Nov 16 16:22 file39
-rw-r--r-- 2 root root 0 Nov 16 16:22 file45
-rw-r--r-- 2 root root 0 Nov 16 16:22 file5
-rw-r--r-- 2 root root 0 Nov 16 16:22 file51
-rw-r--r-- 2 root root 0 Nov 16 16:22 file52
-rw-r--r-- 2 root root 0 Nov 16 16:22 file6
-rw-r--r-- 2 root root 0 Nov 16 16:22 file60
-rw-r--r-- 2 root root 0 Nov 16 16:22 file61
-rw-r--r-- 2 root root 0 Nov 16 16:22 file65
-rw-r--r-- 2 root root 0 Nov 16 16:22 file67
-rw-r--r-- 2 root root 0 Nov 16 16:22 file68
-rw-r--r-- 2 root root 0 Nov 16 16:22 file70
-rw-r--r-- 2 root root 0 Nov 16 16:22 file72
-rw-r--r-- 2 root root 0 Nov 16 16:22 file76
-rw-r--r-- 2 root root 0 Nov 16 16:22 file78
-rw-r--r-- 2 root root 0 Nov 16 16:22 file8
-rw-r--r-- 2 root root 0 Nov 16 16:22 file81
-rw-r--r-- 2 root root 0 Nov 16 16:22 file82
-rw-r--r-- 2 root root 0 Nov 16 16:22 file83
-rw-r--r-- 2 root root 0 Nov 16 16:22 file89
-rw-r--r-- 2 root root 0 Nov 16 16:22 file92
-rw-r--r-- 2 root root 0 Nov 16 16:22 file94
-rw-r--r-- 2 root root 0 Nov 16 16:22 file95
-rw-r--r-- 2 root root 0 Nov 16 16:22 file96
-rw-r--r-- 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b1_2:
total 140
-rw-r--r-- 2 root root 0 Nov 16 16:22 file14
-rw-r--r-- 2 root root 0 Nov 16 16:22 file15
-rw-r--r-- 2 root root 0 Nov 16 16:22 file20
-rw-r--r-- 2 root root 0 Nov 16 16:22 file21
-rw-r--r-- 2 root root 0 Nov 16 16:22 file22
-rw-r--r-- 2 root root 0 Nov 16 16:22 file23
-rw-r--r-- 2 root root 0 Nov 16 16:22 file26
-rw-r--r-- 2 root root 0 Nov 16 16:22 file30
-rw-r--r-- 2 root root 0 Nov 16 16:22 file34
-rw-r--r-- 2 root root 0 Nov 16 16:22 file35
-rw-r--r-- 2 root root 0 Nov 16 16:22 file39
-rw-r--r-- 2 root root 0 Nov 16 16:22 file45
-rw-r--r-- 2 root root 0 Nov 16 16:22 file5
-rw-r--r-- 2 root root 0 Nov 16 16:22 file51
-rw-r--r-- 2 root root 0 Nov 16 16:22 file52
-rw-r--r-- 2 root root 0 Nov 16 16:22 file6
-rw-r--r-- 2 root root 0 Nov 16 16:22 file60
-rw-r--r-- 2 root root 0 Nov 16 16:22 file61
-rw-r--r-- 2 root root 0 Nov 16 16:22 file65
-rw-r--r-- 2 root root 0 Nov 16 16:22 file67
-rw-r--r-- 2 root root 0 Nov 16 16:22 file68
-rw-r--r-- 2 root root 0 Nov 16 16:22 file70
-rw-r--r-- 2 root root 0 Nov 16 16:22 file72
-rw-r--r-- 2 root root 0 Nov 16 16:22 file76
-rw-r--r-- 2 root root 0 Nov 16 16:22 file78
-rw-r--r-- 2 root root 0 Nov 16 16:22 file8
-rw-r--r-- 2 root root 0 Nov 16 16:22 file81
-rw-r--r-- 2 root root 0 Nov 16 16:22 file82
-rw-r--r-- 2 root root 0 Nov 16 16:22 file83
-rw-r--r-- 2 root root 0 Nov 16 16:22 file89
-rw-r--r-- 2 root root 0 Nov 16 16:22 file92
-rw-r--r-- 2 root root 0 Nov 16 16:22 file94
-rw-r--r-- 2 root root 0 Nov 16 16:22 file95
-rw-r--r-- 2 root root 0 Nov 16 16:22 file96
-rw-r--r-- 2 root root 0 Nov 16 16:22 file99

/opt/volume_test/tier_vol/b2_1:
total 104
-rw-r--r-- 2 root root 0 Nov 16 16:22 file10
-rw-r--r-- 2 root root 0 Nov 16 16:22 file100
-rw-r--r-- 2 root root 0 Nov 16 16:22 file12
-rw-r--r-- 2 root root 0 Nov 16 16:22 file13
-rw-r--r-- 2 root root 0 Nov 16 16:22 file18
-rw-r--r-- 2 root root 0 Nov 16 16:22 file28
-rw-r--r-- 2 root root 0 Nov 16 16:22 file3
-rw-r--r-- 2 root root 0 Nov 16 16:22 file32
-rw-r--r-- 2 root root 0 Nov 16 16:22 file37
-rw-r--r-- 2 root root 0 Nov 16 16:22 file4
-rw-r--r-- 2 root root 0 Nov 16 16:22 file40
-rw-r--r-- 2 root root 0 Nov 16 16:22 file43
-rw-r--r-- 2 root root 0 Nov 16 16:22 file46
-rw-r--r-- 2 root root 0 Nov 16 16:22 file47
-rw-r--r-- 2 root root 0 Nov 16 16:22 file48
-rw-r--r-- 2 root root 0 Nov 16 16:22 file50
-rw-r--r-- 2 root root 0 Nov 16 16:22 file54
-rw-r--r-- 2 root root 0 Nov 16 16:22 file55
-rw-r--r-- 2 root root 0 Nov 16 16:22 file58
-rw-r--r-- 2 root root 0 Nov 16 16:22 file66
-rw-r--r-- 2 root root 0 Nov 16 16:22 file7
-rw-r--r-- 2 root root 0 Nov 16 16:22 file71
-rw-r--r-- 2 root root 0 Nov 16 16:22 file75
-rw-r--r-- 2 root root 0 Nov 16 16:22 file86
-rw-r--r-- 2 root root 0 Nov 16 16:22 file9
-rw-r--r-- 2 root root 0 Nov 16 16:22 file91

/opt/volume_test/tier_vol/b2_2:
total 104
-rw-r--r-- 2 root root 0 Nov 16 16:22 file10
-rw-r--r-- 2 root root 0 Nov 16 16:22 file100
-rw-r--r-- 2 root root 0 Nov 16 16:22 file12
-rw-r--r-- 2 root root 0 Nov 16 16:22 file13
-rw-r--r-- 2 root root 0 Nov 16 16:22 file18
-rw-r--r-- 2 root root 0 Nov 16 16:22 file28
-rw-r--r-- 2 root root 0 Nov 16 16:22 file3
-rw-r--r-- 2 root root 0 Nov 16 16:22 file32
-rw-r--r-- 2 root root 0 Nov 16 16:22 file37
-rw-r--r-- 2 root root 0 Nov 16 16:22 file4
-rw-r--r-- 2 root root 0 Nov 16 16:22 file40
-rw-r--r-- 2 root root 0 Nov 16 16:22 file43
-rw-r--r-- 2 root root 0 Nov 16 16:22 file46
-rw-r--r-- 2 root root 0 Nov 16 16:22 file47
-rw-r--r-- 2 root root 0 Nov 16 16:22 file48
-rw-r--r-- 2 root root 0 Nov 16 16:22 file50
-rw-r--r-- 2 root root 0 Nov 16 16:22 file54
-rw-r--r-- 2 root root 0 Nov 16 16:22 file55
-rw-r--r-- 2 root root 0 Nov 16 16:22 file58
-rw-r--r-- 2 root root 0 Nov 16 16:22 file66
-rw-r--r-- 2 root root 0 Nov 16 16:22 file7
-rw-r--r-- 2 root root 0 Nov 16 16:22 file71
-rw-r--r-- 2 root root 0 Nov 16 16:22 file75
-rw-r--r-- 2 root root 0 Nov 16 16:22 file86
-rw-r--r-- 2 root root 0 Nov 16 16:22 file9
-rw-r--r-- 2 root root 0 Nov 16 16:22 file91

/opt/volume_test/tier_vol/b3_1:
total 160
-rw-r--r-- 2 root root 0 Nov 16 16:22 file1
-rw-r--r-- 2 root root 0 Nov 16 16:22 file11
-rw-r--r-- 2 root root 0 Nov 16 16:22 file16
-rw-r--r-- 2 root root 0 Nov 16 16:22 file17
-rw-r--r-- 2 root root 0 Nov 16 16:22 file19
-rw-r--r-- 2 root root 0 Nov 16 16:22 file2
-rw-r--r-- 2 root root 0 Nov 16 16:22 file24
-rw-r--r-- 2 root root 0 Nov 16 16:22 file25
-rw-r--r-- 2 root root 0 Nov 16 16:22 file27
-rw-r--r-- 2 root root 0 Nov 16 16:22 file29
-rw-r--r-- 2 root root 0 Nov 16 16:22 file31
-rw-r--r-- 2 root root 0 Nov 16 16:22 file33
-rw-r--r-- 2 root root 0 Nov 16 16:22 file36
-rw-r--r-- 2 root root 0 Nov 16 16:22 file38
-rw-r--r-- 2 root root 0 Nov 16 16:22 file41
-rw-r--r-- 2 root root 0 Nov 16 16:22 file42
-rw-r--r-- 2 root root 0 Nov 16 16:22 file44
-rw-r--r-- 2 root root 0 Nov 16 16:22 file49
-rw-r--r-- 2 root root 0 Nov 16 16:22 file53
-rw-r--r-- 2 root root 0 Nov 16 16:22 file56
-rw-r--r-- 2 root root 0 Nov 16 16:22 file57
-rw-r--r-- 2 root root 0 Nov 16 16:22 file59
-rw-r--r-- 2 root root 0 Nov 16 16:22 file62
-rw-r--r-- 2 root root 0 Nov 16 16:22 file63
-rw-r--r-- 2 root root 0 Nov 16 16:22 file64
-rw-r--r-- 2 root root 0 Nov 16 16:22 file69
-rw-r--r-- 2 root root 0 Nov 16 16:22 file73
-rw-r--r-- 2 root root 0 Nov 16 16:22 file74
-rw-r--r-- 2 root root 0 Nov 16 16:22 file77
-rw-r--r-- 2 root root 0 Nov 16 16:22 file79
-rw-r--r-- 2 root root 0 Nov 16 16:22 file80
-rw-r--r-- 2 root root 0 Nov 16 16:22 file84
-rw-r--r-- 2 root root 0 Nov 16 16:22 file85
-rw-r--r-- 2 root root 0 Nov 16 16:22 file87
-rw-r--r-- 2 root root 0 Nov 16 16:22 file88
-rw-r--r-- 2 root root 0 Nov 16 16:22 file90
-rw-r--r-- 2 root root 0 Nov 16 16:22 file93
-rw-r--r-- 2 root root 0 Nov 16 16:22 file97
-rw-r--r-- 3 root root 0 Nov 16 16:22 file98
-rw-r--r-- 3 root root 0 Nov 16 16:22 file98_link

/opt/volume_test/tier_vol/b3_2:
total 160
-rw-r--r-- 2 root root 0 Nov 16 16:22 file1
-rw-r--r-- 2 root root 0 Nov 16 16:22 file11
-rw-r--r-- 2 root root 0 Nov 16 16:22 file16
-rw-r--r-- 2 root root 0 Nov 16 16:22 file17
-rw-r--r-- 2 root root 0 Nov 16 16:22 file19
-rw-r--r-- 2 root root 0 Nov 16 16:22 file2
-rw-r--r-- 2 root root 0 Nov 16 16:22 file24
-rw-r--r-- 2 root root 0 Nov 16 16:22 file25
-rw-r--r-- 2 root root 0 Nov 16 16:22 file27
-rw-r--r-- 2 root root 0 Nov 16 16:22 file29
-rw-r--r-- 2 root root 0 Nov 16 16:22 file31
-rw-r--r-- 2 root root 0 Nov 16 16:22 file33
-rw-r--r-- 2 root root 0 Nov 16 16:22 file36
-rw-r--r-- 2 root root 0 Nov 16 16:22 file38
-rw-r--r-- 2 root root 0 Nov 16 16:22 file41
-rw-r--r-- 2 root root 0 Nov 16 16:22 file42
-rw-r--r-- 2 root root 0 Nov 16 16:22 file44
-rw-r--r-- 2 root root 0 Nov 16 16:22 file49
-rw-r--r-- 2 root root 0 Nov 16 16:22 file53
-rw-r--r-- 2 root root 0 Nov 16 16:22 file56
-rw-r--r-- 2 root root 0 Nov 16 16:22 file57
-rw-r--r-- 2 root root 0 Nov 16 16:22 file59
-rw-r--r-- 2 root root 0 Nov 16 16:22 file62
-rw-r--r-- 2 root root 0 Nov 16 16:22 file63
-rw-r--r-- 2 root root 0 Nov 16 16:22 file64
-rw-r--r-- 2 root root 0 Nov 16 16:22 file69
-rw-r--r-- 2 root root 0 Nov 16 16:22 file73
-rw-r--r-- 2 root root 0 Nov 16 16:22 file74
-rw-r--r-- 2 root root 0 Nov 16 16:22 file77
-rw-r--r-- 2 root root 0 Nov 16 16:22 file79
-rw-r--r-- 2 root root 0 Nov 16 16:22 file80
-rw-r--r-- 2 root root 0 Nov 16 16:22 file84
-rw-r--r-- 2 root root 0 Nov 16 16:22 file85
-rw-r--r-- 2 root root 0 Nov 16 16:22 file87
-rw-r--r-- 2 root root 0 Nov 16 16:22 file88
-rw-r--r-- 2 root root 0 Nov 16 16:22 file90
-rw-r--r-- 2 root root 0 Nov 16 16:22 file93
-rw-r--r-- 2 root root 0 Nov 16 16:22 file97
-rw-r--r-- 3 root root 0 Nov 16 16:22 file98
-rw-r--r-- 3 root root 0 Nov 16 16:22 file98_link

/opt/volume_test/tier_vol/b4_1:
total 0

/opt/volume_test/tier_vol/b4_2:
total 0

/opt/volume_test/tier_vol/b5_1:
total 0

/opt/volume_test/tier_vol/b5_2:
total 0
[root@gfvm3 tierd_volume]#
===============================================================================
Comment 7 Vivek Agarwal 2015-11-24 10:04:56 UTC 
https://code.engineering.redhat.com/gerrit/62104
Comment 8 Nag Pavan Chilakam 2015-11-24 11:38:32 UTC 
Verified on glusterfs-3.7.5-7:
============================
1)both in test and cache mode 
2)on nfs and fuse
3)while promotes and demotes were happening, the ls -l |wc -l was giving a constant number of files as the count. 

I had about 8000 files and heated some and some got cooled. While they were migrating issued ll|wc -l and was getting constantly the right number of files

[root@zod ~]# gluster v tier testmode status;gluster v rebal testmode status
Node                 Promoted files       Demoted files        Status              
---------            ---------            ---------            ---------           
localhost            3424                 0                    in progress         
10.70.34.43          0                    7234                 in progress         
Tiering Migration Functionality: testmode: success
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost             7225        0Bytes         20464             0             0          in progress             626.00
                             10.70.34.43            14134        0Bytes         34876             0             0          in progress             626.00
volume rebalance: testmode: success
[root@zod ~]# gluster v tier testmode status;gluster v rebal testmode status
Node                 Promoted files       Demoted files        Status              
---------            ---------            ---------            ---------           
localhost            3444                 0                    in progress         
10.70.34.43          0                    7234                 in progress         
Tiering Migration Functionality: testmode: success
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost             7244        0Bytes         20464             0             0          in progress             627.00
                             10.70.34.43            14134        0Bytes         34876             0             0          in progress             627.00
volume rebalance: testmode: success
[root@zod ~]# 



NFS MOunt:
[root@dhcp35-214 mnt]# cd testmode/
[root@dhcp35-214 testmode]# ll |wc -l
8001
[root@dhcp35-214 testmode]# ll |wc -l
8001
[root@dhcp35-214 testmode]# ll |wc -l
8001
[root@dhcp35-214 testmode]# 

Fuse mount:

[root@localhost testmode]# for i in {1..3000};do echo "hi" >> h$i;done;ll|wc -l
8001
Comment 9 Dan Lambright 2015-11-24 13:21:14 UTC 
*** Bug 1271947 has been marked as a duplicate of this bug. ***
Comment 11 errata-xmlrpc 2016-03-01 05:47:08 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0193.html