Bug 1121584 - remove-brick stop & status not validating the bricks to check whether the rebalance is actually started on them
Summary: remove-brick stop & status not validating the bricks to check whether the reb...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: mainline
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Gaurav Kumar Garg
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1121585
TreeView+ depends on / blocked
 
Reported: 2014-07-21 09:41 UTC by SATHEESARAN
Modified: 2016-06-05 23:37 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.7.0
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1121585 (view as bug list)
Environment:
gluster-cli
Last Closed: 2015-05-14 17:26:24 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description SATHEESARAN 2014-07-21 09:41:17 UTC
Description of problem:
-----------------------
remove-brick stop & remove-brick status was not validating whether the rebalance is actually triggered on those bricks. The ill-effect of this missing check are,
1. 'remove-brick status' shows output for non-existing bricks
2. 'remove-brick stop' stops in-progress rebalance, though provided with non-existent brick

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
mainline

How reproducible:
------------------
always

Steps to Reproduce:
-------------------
1. create a distribute volume with 2 or more bricks (say brick1,brick2,brick3)
2. perform 'remove-brick start' on the volume, for brick3
3. Get the 'remove-brick status' on the volume, but with non-existent brick ( say brick4 )

Notice the output: Though the brick is non-existent, you get rebalance status of last rebalance

4. Stop the remove-brick on the volume for non-existent brick ( say brick4 )

Notice the result: The currently running rebalance would be stopped, though non-existent brick is supplied for remove-brick stop

Actual results:
---------------
'remove-brick status' & 'remove-brick stop' works even when supplied with non-existent bricks or brick which never took part in remove-brick


Expected results:
-----------------
'remove-brick status' & 'remove-brick stop' should throw proper error message in 2 cases
1. when non-existent brick is provided
2. Bricks those weren't participating in earlier remove-brick operation at all
e.g. if remove-brick is performed on brick1 & brick2, remove-brick stop & status could only performed on brick1 & brick2 and not on brick3 & brick4

Additional info:
----------------

[root@10.70.37.136:~ ] # gluster v i
 
Volume Name: dv
Type: Distribute
Volume ID: 6b811482-bc48-4142-b988-2b8efe7cae46
Status: Started
Snap Volume: no
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 10.70.37.136:/rhs/brick1/b1       
Brick2: 10.70.37.136:/rhs/brick2/b2         
Brick3: 10.70.37.136:/tmp/1
Options Reconfigured:
performance.readdir-ahead: on
auto-delete: disable
snap-max-soft-limit: 90
snap-max-hard-limit: 256

[root@10.70.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/1 start
volume remove-brick start: success
ID: 5e880ba6-d530-42c5-a665-feab98fc2596

<<< using the non-existent brick for remove-brick stop >>>
[Mon Jul 21 14:30:55 UTC 2014 root@10.70.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/2 stop
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00
'remove-brick' process may be in the middle of a file migration.
The process will be fully stopped once the migration of the file is complete.
Please check remove-brick process for completion before doing any further brick related tasks on the volume.

[root@10.70.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/1 start
volume remove-brick start: success
ID: eb15de2a-3639-4f3d-b932-d1c0783b72e6

[root@10.70.37.136:~ ] # gluster volume remove-brick dv 10.70.37.136:/tmp/2 status
                                    Node Rebalanced-files          size       scanned      failures       skipped               status   run time in secs
                               ---------      -----------   -----------   -----------   -----------   -----------         ------------     --------------
                               localhost                0        0Bytes             0             0             0            completed               0.00

Comment 1 SATHEESARAN 2014-07-21 10:11:27 UTC
Thanks Gaurav for uncovering this issue

Comment 2 Anand Avati 2015-02-18 13:09:51 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#1) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 3 Anand Avati 2015-02-19 06:50:58 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#2) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 4 Anand Avati 2015-02-19 12:44:42 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#3) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 5 Anand Avati 2015-02-20 06:27:48 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#4) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 6 Anand Avati 2015-03-04 07:07:07 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#5) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 7 Anand Avati 2015-03-04 09:39:18 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#6) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 8 Anand Avati 2015-03-24 08:37:29 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#7) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 9 Anand Avati 2015-03-24 11:26:29 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#8) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 10 Anand Avati 2015-03-25 12:24:06 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#9) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 11 Anand Avati 2015-03-27 09:33:56 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#10) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 12 Anand Avati 2015-03-30 12:51:33 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#11) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 13 Anand Avati 2015-04-02 06:19:01 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#12) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 14 Anand Avati 2015-04-02 10:34:42 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#13) for review on master by Vijay Bellur (vbellur@redhat.com)

Comment 15 Anand Avati 2015-04-09 12:12:26 UTC
REVIEW: http://review.gluster.org/9681 (glusterd: remove-brick status/stop should not show output for non-existing brick) posted (#14) for review on master by Gaurav Kumar Garg (ggarg@redhat.com)

Comment 16 Anand Avati 2015-04-09 17:22:20 UTC
COMMIT: http://review.gluster.org/9681 committed in master by Krishnan Parthasarathi (kparthas@redhat.com) 
------
commit 2788ddd3a0afa98f78128247cca89427a323b090
Author: Gaurav Kumar Garg <ggarg@redhat.com>
Date:   Sun Feb 15 19:22:13 2015 +0530

    glusterd: remove-brick status/stop should not show output for non-existing brick
    
    Previously when user start remove-brick operation on a volume then by
    giving non-existing brick for remove-brick status/stop command it was
    showing remove-brick status/stoping remove-brick operation on a volume.
    
    With this fix it will validate bricks which user have given for
    remove-brick status/stop command and if bricks are part of volume then
    it will show statistics of remove-brick operation otherwise it will show
    error "Incorrect brick <brick_name> for <volume_name>".
    
    Change-Id: I151284ef78c25f52d1b39cdbd71ebfb9eb4b8471
    BUG: 1121584
    Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
    Reviewed-on: http://review.gluster.org/9681
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
    Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>

Comment 17 Niels de Vos 2015-05-14 17:26:24 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 18 Niels de Vos 2015-05-14 17:35:29 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 19 Niels de Vos 2015-05-14 17:37:51 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 20 Niels de Vos 2015-05-14 17:42:42 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.