+++ This bug was initially created as a clone of Bug #1349284 +++ +++ This bug was initially created as a clone of Bug #1299841 +++ Description of problem: Today, When a file is heated which is of size greater than the size of hot tier's high water mark, the file is promoted. Gluster should intelligently determine that the size of the file is more than high watermark and should not get promoted. with the following scenario, hot tier capacity: 100Gb high water mark - 70% i.e., 70Gb File is created of size 80Gb Having such file in hot tier has multiple consequences. 1) When this file is in hot tier, we always exceed high watermark and get into complete demotion mode. As a result, we make all other on hot tier a possible candidate for demotion. 2) No other files will be promoted until this file moves to cold tier 3) Possibility of moving such a huge file back and forth from cold to hot tier which might affect performance Version-Release number of selected component (if applicable): glusterfs-3.7.5-15.el7rhgs.x86_64 How reproducible: Always Steps to Reproduce: 1. create a gluster volume 2. create a 80Gb file, say file1 3. Attach hot tier of size 100Gb and set high watermark at 70% 4. heat the file - file1 created in step 2 Actual results: File gets promoted Expected results: File should not get promoted Additional info: --- Additional comment from Red Hat Bugzilla Rules Engine on 2016-01-19 06:38:14 EST --- This bug is automatically being proposed for the current z-stream release of Red Hat Gluster Storage 3 by setting the release flag 'rhgs‑3.1.z' to '?'. If this bug should be proposed for a different release, please manually change the proposed release flag. --- Additional comment from Vijay Bellur on 2016-06-23 03:18:47 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote estimated block consumption > hi watermark) posted (#1) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-06-23 03:59:25 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote estimated block consumption > hi watermark) posted (#2) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-06-29 06:31:50 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#3) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-05 06:27:23 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#4) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-05 12:40:26 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#5) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-12 02:44:12 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#6) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-12 09:06:15 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#7) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-15 04:57:49 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#8) for review on master by Milind Changire (mchangir) --- Additional comment from Mohammed Rafi KC on 2016-07-15 06:33:30 EDT --- upstream master patch : http://review.gluster.org/#/c/14780/ --- Additional comment from Vijay Bellur on 2016-07-19 06:54:31 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#9) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-20 02:52:21 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#10) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-20 09:04:04 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#11) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-21 03:30:03 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#12) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-27 09:54:35 EDT --- REVIEW: http://review.gluster.org/14780 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#13) for review on master by Milind Changire (mchangir) --- Additional comment from Vijay Bellur on 2016-07-29 10:14:40 EDT --- COMMIT: http://review.gluster.org/14780 committed in master by Dan Lambright (dlambrig) ------ commit 1f4e41e8c2f5f4af4564caba0a08996853f089f4 Author: Milind Changire <mchangir> Date: Thu Jun 23 12:37:51 2016 +0530 cluster/tier: dont promote if estimated block consumption > hi watermark Add test to fail promotion if estimated block consumption grows beyond hi watermark. Skip file migrations until next cycle if tier_get_fs_stat() fails in tier_migrate_using_query_file() Change-Id: Ice04572fa739c09109c4433e65965197482a7beb BUG: 1349284 Signed-off-by: Milind Changire <mchangir> Reviewed-on: http://review.gluster.org/14780 Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: mohammed rafi kc <rkavunga> Reviewed-by: Dan Lambright <dlambrig>
REVIEW: http://review.gluster.org/15065 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#1) for review on release-3.8 by Milind Changire (mchangir)
REVIEW: http://review.gluster.org/15065 (cluster/tier: dont promote if estimated block consumption > hi watermark) posted (#2) for review on release-3.8 by Milind Changire (mchangir)
COMMIT: http://review.gluster.org/15065 committed in release-3.8 by Niels de Vos (ndevos) ------ commit 14b3a145a31e9f6760fb41bb104fcc2cf3bcc463 Author: Milind Changire <mchangir> Date: Thu Jun 23 12:37:51 2016 +0530 cluster/tier: dont promote if estimated block consumption > hi watermark Add test to fail promotion if estimated block consumption grows beyond hi watermark. Skip file migrations until next cycle if tier_get_fs_stat() fails in tier_migrate_using_query_file() > Reviewed-on: http://review.gluster.org/14780 > Smoke: Gluster Build System <jenkins.org> > CentOS-regression: Gluster Build System <jenkins.org> > NetBSD-regression: NetBSD Build System <jenkins.org> > Reviewed-by: mohammed rafi kc <rkavunga> > Reviewed-by: Dan Lambright <dlambrig> (cherry picked from commit 1f4e41e8c2f5f4af4564caba0a08996853f089f4) Change-Id: Ice04572fa739c09109c4433e65965197482a7beb BUG: 1362198 Signed-off-by: Milind Changire <mchangir> Reviewed-on: http://review.gluster.org/15065 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Niels de Vos <ndevos>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.2, please open a new bug report. glusterfs-3.8.2 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://www.gluster.org/pipermail/announce/2016-August/000058.html [2] https://www.gluster.org/pipermail/gluster-users/