We need a way to pause tiering to take snapshot. A file cannot be in the middle of promotion/demotion when a snapshot is taken. If a file is in the middle of migration, abort the migration and rollback any in-progress work. Implement a CLI for this and to resume migration.
REVIEW: http://review.gluster.org/12304 (cluster/tier: WIP add pause tier feature (required for snapshots)) posted (#3) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12304 (cluster/tier: add pause tier for snapshots) posted (#4) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12304 (cluster/tier: add pause tier for snapshots) posted (#5) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12304 (cluster/tier: add pause tier for snapshots) posted (#6) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12304 (cluster/tier: add pause tier for snapshots) posted (#7) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12304 (cluster/tier: add pause tier for snapshots) posted (#8) for review on master by Dan Lambright (dlambrig)
COMMIT: http://review.gluster.org/12304 committed in master by Dan Lambright (dlambrig) ------ commit 36974c36fa4231df3f0e9428a9da6d1aa33348ab Author: Dan Lambright <dlambrig> Date: Mon Oct 5 19:52:02 2015 +0000 cluster/tier: add pause tier for snapshots Snaps of tiered volumes cannot handle files undergoing migration. We implement a helper mechanism to "pause" migration. Any files undergoing migration are aborted. Clean up is done to remove sticky bits and data at the destination. Migration is restarted after snap completes. For testing an internal switch is added. It is not exposed externally. gluster volume set vol1 tier-pause [true|false] Change-Id: Ia85bbf89ac142e9b7e73fcbef98bb9da86097799 BUG: 1267950 Signed-off-by: Dan Lambright <dlambrig> Reviewed-on: http://review.gluster.org/12304 Reviewed-by: N Balachandran <nbalacha> Tested-by: NetBSD Build System <jenkins.org> Tested-by: Gluster Build System <jenkins.com>
REVIEW: http://review.gluster.org/12934 (cluster/tier: fix pause tier) posted (#3) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12934 (cluster/tier: fix pause tier) posted (#4) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12934 (cluster/tier: fix pause tier) posted (#5) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12987 (cluster/tier: do not block in synctask created from pause tier) posted (#1) for review on master by Dan Lambright (dlambrig)
12987 shall supersede 12934 per analysis of code, stack [1], discussions, and ultimately testing the correction over the reproduced original problem. [1] http://ur1.ca/ocgrq
REVIEW: http://review.gluster.org/12987 (cluster/tier: do not block in synctask created from pause tier) posted (#2) for review on master by Dan Lambright (dlambrig)
REVIEW: http://review.gluster.org/12987 (cluster/tier: do not block in synctask created from pause tier) posted (#3) for review on master by Dan Lambright (dlambrig)
COMMIT: http://review.gluster.org/12987 committed in master by Dan Lambright (dlambrig) ------ commit 3a094f1de03b3da8cdff650c14e46aab87e1905b Author: Dan Lambright <dlambrig> Date: Wed Dec 16 15:28:57 2015 -0500 cluster/tier: do not block in synctask created from pause tier We had run sleep() in the pause tier callback. Blocking within a synctask is dangerous. The sleep() call does not inform the synctask scheduler that a thread is no longer running. It therefore believes it is running. If a second synctask already exists, it may not be able to run. This occurs if the thread limit in the pool has been reached. Note the pool size only grows when a synctask is created, not when it is moved from wait state to run state, as is the case when an FOP completes. When the tier is paused during migration, synctasks already exist waiting for responses to FOPs to the server with high probability. The fix is to yield() in the RPC callback, which will place the synctask into the wait queue and free up a thread for the FOP callback. A timer wakes the callback after sufficient time has elapsed for the pause to occur. Change-Id: I6a947ee04c6e5649946cb6d8207ba17263a67fc6 BUG: 1267950 Signed-off-by: Dan Lambright <dlambrig> Reviewed-on: http://review.gluster.org/12987 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Rajesh Joseph <rjoseph>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report. glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/ [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user