Description of problem: Currently all of brick, gsyncd and other glustershd services/processes are killed without checking for ongoing tasks such as geo-rep, self-heal, rebalance and etc. which may lead to inconsistency after the node is brought back. Version-Release number of selected component (if applicable): mainline Actual results: inconsistency in data Expected results: Smooth resume with no data inconsistency
REVIEW: http://review.gluster.org/15187 (extras: kill gsyncd before glusterfsd) posted (#1) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15188 (extras: kill processes gracefully) posted (#1) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15187 (extras: kill gsyncd before glusterfsd) posted (#2) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15188 (extras: kill processes gracefully) posted (#2) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15188 (extras: kill processes gracefully) posted (#3) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15188 (extras: kill processes gracefully) posted (#4) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15188 (extras: kill processes gracefully) posted (#5) for review on master by Prasanna Kumar Kalever (pkalever)
REVIEW: http://review.gluster.org/15188 (extras: kill processes gracefully) posted (#6) for review on master by Prasanna Kumar Kalever (pkalever)
COMMIT: http://review.gluster.org/15188 committed in master by Atin Mukherjee (amukherj) ------ commit 14b46be09c98c1ef75f55af34ec762b3e34899f1 Author: Prasanna Kumar Kalever <prasanna.kalever> Date: Wed Aug 17 17:31:17 2016 +0530 extras: kill processes gracefully Currently all of brick, gsyncd and other glustershd services/processes are killed without checking for ongoing tasks such as geo-rep, self-heal, rebalance and etc. which may lead to inconsistency after the node is brought back. This patch introduce an option '-g' which ensures whether all the gluster processes are ready (not busy) to be terminated before we executing 'kill' on them Usage: ./extras/stop-all-gluster-processes.sh [-g] [-h] options: -g Terminate in graceful mode -h Show this message, then exit eg: 1. ./extras/stop-all-gluster-processes.sh 2. ./extras/stop-all-gluster-processes.sh -g By default, this script executes in force mode, processes are killed without checking for ongoing tasks, on specifying '-g' option this script works in graceful mode, which returns exitcode if some of gluster processes are busy in doing their jobs. exitcodes include: 0 No errors/Success 64 Rebalance is in progress 65 Self-Heal is in progress 66 Tier daemon running on this node 127 option not found Change-Id: I2f924b2bf9f04a81788d0f5604895a42755b33a1 BUG: 1367771 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever> Reviewed-on: http://review.gluster.org/15188 Tested-by: Prasanna Kumar Kalever <pkalever> Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Rajesh Joseph <rjoseph> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Ravishankar N <ravishankar> Reviewed-by: mohammed rafi kc <rkavunga> Reviewed-by: Atin Mukherjee <amukherj>
COMMIT: http://review.gluster.org/15187 committed in master by Atin Mukherjee (amukherj) ------ commit 87303b77ddbdc29a415acc2c73bfaafa9be10609 Author: Prasanna Kumar Kalever <prasanna.kalever> Date: Wed Aug 17 17:47:08 2016 +0530 extras: kill gsyncd before glusterfsd It's a good practice to kill gsyncd before brick process, else we see unnecessary loops and logs of bring up the faulty bricks Change-Id: I2d2176f2bf5014f3afd622194a8b2d60c86926af BUG: 1367771 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever> Reviewed-on: http://review.gluster.org/15187 Tested-by: Prasanna Kumar Kalever <pkalever> CentOS-regression: Gluster Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Atin Mukherjee <amukherj> Reviewed-by: Kotresh HR <khiremat>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.0, please open a new bug report. glusterfs-3.9.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/gluster-users/2016-November/029281.html [2] https://www.gluster.org/pipermail/gluster-users/