Description of problem: ----------------------- remove-brick stop & remove-brick status was not validating whether the rebalance is actually triggered on those bricks. The ill-effect of this missing check are, 1. 'remove-brick status' shows output for non-existing bricks 2. 'remove-brick stop' stops in-progress rebalance, though provided with non-existent brick Version-Release number of selected component (if applicable): ------------------------------------------------------------- mainline How reproducible: ------------------ always Steps to Reproduce: ------------------- 1. create a distribute volume with 2 or more bricks (say brick1,brick2,brick3) 2. perform 'remove-brick start' on the volume, for brick3 3. Get the 'remove-brick status' on the volume, but with non-existent brick ( say brick4 ) Notice the output: Though the brick is non-existent, you get rebalance status of last rebalance 4. Stop the remove-brick on the volume for non-existent brick ( say brick4 ) Notice the result: The currently running rebalance would be stopped, though non-existent brick is supplied for remove-brick stop Actual results: --------------- 'remove-brick status' & 'remove-brick stop' works even when supplied with non-existent bricks or brick which never took part in remove-brick Expected results: ----------------- 'remove-brick status' & 'remove-brick stop' should throw proper error message in 2 cases 1. when non-existent brick is provided 2. Bricks those weren't participating in earlier remove-brick operation at all e.g. if remove-brick is performed on brick1 & brick2, remove-brick stop & status could only performed on brick1 & brick2 and not on brick3 & brick4 Additional info: ---------------- [root.37.136:~ ] # gluster v i Volume Name: dv Type: Distribute Volume ID: 6b811482-bc48-4142-b988-2b8efe7cae46 Status: Started Snap Volume: no Number of Bricks: 3 Transport-type: tcp Bricks: Brick1: 10.70.37.136:/rhs/brick1/b1 Brick2: 10.70.37.136:/rhs/brick2/b2 Brick3: 10.70.37.136:/tmp/1 Options Reconfigured: performance.readdir-ahead: on auto-delete: disable snap-max-soft-limit: 90 snap-max-hard-limit: 256 [root.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/1 start volume remove-brick start: success ID: 5e880ba6-d530-42c5-a665-feab98fc2596 <<< using the non-existent brick for remove-brick stop >>> [Mon Jul 21 14:30:55 UTC 2014 root.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/2 stop Node Rebalanced-files size scanned failures skipped status run time in secs --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 0 0Bytes 0 0 0 completed 0.00 'remove-brick' process may be in the middle of a file migration. The process will be fully stopped once the migration of the file is complete. Please check remove-brick process for completion before doing any further brick related tasks on the volume. [root.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/1 start volume remove-brick start: success ID: eb15de2a-3639-4f3d-b932-d1c0783b72e6 [root.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/2 status Node Rebalanced-files size scanned failures skipped status run time in secs --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 0 0Bytes 0 0 0 completed 0.00
Thanks Gaurav for uncovering this issue
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#1) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#2) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#3) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#4) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#5) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#6) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#7) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#8) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#9) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#10) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#11) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#12) for review on master by Gaurav Kumar Garg (ggarg)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#13) for review on master by Vijay Bellur (vbellur)
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#14) for review on master by Gaurav Kumar Garg (ggarg)
COMMIT: http://review.gluster.org/9681 committed in master by Krishnan Parthasarathi (kparthas) ------ commit 2788ddd3a0afa98f78128247cca89427a323b090 Author: Gaurav Kumar Garg <ggarg> Date: Sun Feb 15 19:22:13 2015 +0530 glusterd: remove-brick status/stop should not show output for non-existing brick Previously when user start remove-brick operation on a volume then by giving non-existing brick for remove-brick status/stop command it was showing remove-brick status/stoping remove-brick operation on a volume. With this fix it will validate bricks which user have given for remove-brick status/stop command and if bricks are part of volume then it will show statistics of remove-brick operation otherwise it will show error "Incorrect brick <brick_name> for <volume_name>". Change-Id: I151284ef78c25f52d1b39cdbd71ebfb9eb4b8471 BUG: 1121584 Signed-off-by: Gaurav Kumar Garg <ggarg> Reviewed-on: http://review.gluster.org/9681 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Krishnan Parthasarathi <kparthas> Tested-by: Krishnan Parthasarathi <kparthas>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report. glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939 [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user