Description of problem: Detach tier commit is allowed to succeed before detach tier is complete and as a result files are left behind in hot tier. [root@dhcp47-187 glusterfs]# gluster v tier spiderman detach start volume detach-tier start: success ID: 06d6e746-a1a7-4c8c-b4bc-52b261d0d0b4 [root@dhcp47-187 glusterfs]# gluster v tier spiderman detach commit Removing tier can result in data loss. Do you want to Continue? (y/n) y volume detach-tier commit: success Check the detached bricks to ensure all files are migrated. If files with data are found on the brick path, copy them via a gluster mount point before re-purposing the removed brick. Version-Release number of selected component (if applicable): glusterfs-server-3.7.9-4.el7rhgs.x86_64 How reproducible: Always Steps to Reproduce: 1. create a tiered volume 2. write files to hot tier, write enough files so that tier-rebalance operation takes a while to migrate files from hot tier to cold tier 3. Perform detach tier start and immediately perform a detach tier commit Actual results: Files are left unmigrated in hot tier and detach tier commit succeeds Expected results: detach tier commit should fail until detach tier operation moves to completed state, so no files are left behind in hot tier unmigrated Additional info: sosreports shall be attached shortly.
This appears to be a downstream only problem. I could recreate it downstream but not on 3.8 or upstream.
The bug appears to have been that we did not return a failure from glusterd back to the CLI. It appears comment #4 is incorrect and the problem exists upstream as well. I will get a patch upstream, and then get it downstream.
After some further investigation, it appears to have been backported incorrectly downstream. I do not know why this is. At any rate the patch number is 74125.
Downstream patch : https://code.engineering.redhat.com/gerrit/#/c/74126/ 3.7 patch : http://review.gluster.org/14327 Master and 3.8 release branch doesn't have this issue.
The issue reported in the bug is no more seen in build - glusterfs-3.7.9-5. when detach tier commit is run while detach tier is in progress, it fails with an error message # gluster v tier spiderman detach commit Removing tier can result in data loss. Do you want to Continue? (y/n) y volume detach-tier commit: failed: Brick's in Hot tier is not decommissioned yet. Use gluster volume detach-tier <VOLNAME> <start | commit | force> command instead Tier command failed This bug will be moved to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1240