Bug 1271151 - Data Tiering:Files not getting migrated from hot to cold when the hot brick is full
Summary: Data Tiering:Files not getting migrated from hot to cold when the hot brick i...
Keywords:
Status: CLOSED EOL
Alias: None
Product: GlusterFS
Classification: Community
Component: tiering
Version: 3.7.5
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
Assignee: Nithya Balachandran
QA Contact: bugs@gluster.org
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-13 09:18 UTC by Nag Pavan Chilakam
Modified: 2017-03-08 10:51 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1278391 1278392 (view as bug list)
Environment:
Last Closed: 2017-03-08 10:51:11 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)
cli logs while filing bug (110.93 KB, text/plain)
2015-10-13 10:43 UTC, Nag Pavan Chilakam
no flags Details

Description Nag Pavan Chilakam 2015-10-13 09:18:43 UTC
Description of problem:
=======================
When a brick is full, files are not getting migrated at all.
I created a hot tier with 2x2 with each brick of 1GB size. That means effectively the hot tier is of 2GB size.
I started to create a file of 1.5GB using dd command, which was getting created on 1st brick. But as the 1GB gets filled, the new file stops to grow with below error:
dd: error writing ‘zerobyte.50’: Input/output error
dd: closing output file ‘zerobyte.50’: Input/output error

Now the file is not getting demoted at all and hence no more writes are going to the hot tier.

Similarly, if i fill the other brick too, then it essentially means that the whole hot tier is full.

In this case, no more new creates are being allowed even to the cold tier (also refer bz#1259312)

Version-Release number of selected component (if applicable):
===========================================================
glusterfs-server-3.7.5-0.18.git2def6bb.el7.centos.x86_64






Steps to Reproduce:
==================
1.create a 2x2 vol and start it
2.attach a tier of 2x2 with limited size, say each brick of 1GB.
Also turn on ctr (note in this case quota was not turned on)
3.Now create a file of 1.5GB such that it goes to one of the hot bricks
It will fail after it creates 1GB
4. Now this file doesnt get demoted at all 
5.Now again create a 1.5GB file such that it goes to the other hot brick(to make hot layer full)
5. Now file doenst get demoted at all



Expected results:
================
File must get demoted once it is old and idle

Comment 1 Nag Pavan Chilakam 2015-10-13 09:39:20 UTC
vol name:estonia



sosreports:rhsqe-repo.lab.eng.blr.redhat.com:/home/repo/sosreports/bug.1271151

Comment 2 Nag Pavan Chilakam 2015-10-13 10:43:42 UTC
Created attachment 1082325 [details]
cli logs while filing bug

Comment 3 Dan Lambright 2015-10-29 21:01:33 UTC
DHT does not accept creation of files beyond the brick size. So I am not sure if we can fix the test the way it is structured, but nithya can confirm.

In practice the hot brick would be 10s or 100s of GB. A file created would be a fraction of that size. As files are created on the hot tier it would become more full, until eventually a watermarked was crossed. At that time files would be created on the cold tier.

Comment 4 Kaushal 2017-03-08 10:51:11 UTC
This bug is getting closed because GlusteFS-3.7 has reached its end-of-life.

Note: This bug is being closed using a script. No verification has been performed to check if it still exists on newer releases of GlusterFS.
If this bug still exists in newer GlusterFS releases, please reopen this bug against the newer release.


Note You need to log in before you can comment on or make changes to this bug.