Bug 1421590 - Bricks take up new ports upon volume restart after add-brick op with brick mux enabled
Summary: Bricks take up new ports upon volume restart after add-brick op with brick mu...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: mainline
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
Assignee: Samikshan Bairagya
QA Contact:
URL:
Whiteboard: brick-multiplexing-testing
Depends On:
Blocks: 1427461
TreeView+ depends on / blocked
 
Reported: 2017-02-13 08:31 UTC by Samikshan Bairagya
Modified: 2017-05-30 18:42 UTC (History)
4 users (show)

Fixed In Version: glusterfs-3.11.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1427461 (view as bug list)
Environment:
Last Closed: 2017-05-30 18:42:26 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Samikshan Bairagya 2017-02-13 08:31:12 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce and actual results:

Taking a 1 node cluster here to list down the steps to reproduce, but this can be reproduced on multi-node cluster too.

1. Enable brick multiplexing
2. Create 1 volume with one brick
2. Start the volume and check volume status. The brick will be using port 49152
3. Add a brick to the volume and check vol status. Both bricks use 49152
4. Stop the volume and then start it.
5. Check volume status. Both bricks now use 49153.
6. If you restart the volume again and check the status, the bricks would now use 49154. For every restart, the bricks take up the next port.

Expected results:
The bricks should use the ports being used upon restart and not take up a new port.

Comment 1 Atin Mukherjee 2017-02-13 08:48:38 UTC
Samikshan - just to double check, is this issue not seen if brick mux is disabled?

Comment 2 Samikshan Bairagya 2017-02-13 09:20:47 UTC
(In reply to Atin Mukherjee from comment #1)
> Samikshan - just to double check, is this issue not seen if brick mux is
> disabled?

No. I tested this with brick mux disabled. This issue wasn't seen.

Comment 3 Jeff Darcy 2017-02-13 14:53:07 UTC
We're likely to encounter many of these "grey area" bugs which are not addressed by any existing requirements or tests.  Since fixing them is already likely to become a bottleneck, and manual testing is likely to make that even worse, it would be very helpful if other developers could provide the missing tests.  Any suggestions for how best to do that?

Comment 4 Worker Ant 2017-02-20 13:14:38 UTC
REVIEW: https://review.gluster.org/16689 (core: Clean up pmap registry up correctly on volume/brick stop) posted (#1) for review on master by Samikshan Bairagya (samikshan)

Comment 5 Worker Ant 2017-02-20 14:24:55 UTC
REVIEW: https://review.gluster.org/16689 (core: Clean up pmap registry up correctly on volume/brick stop) posted (#2) for review on master by Samikshan Bairagya (samikshan)

Comment 6 Worker Ant 2017-02-21 14:48:07 UTC
REVIEW: https://review.gluster.org/16689 (core: Clean up pmap registry up correctly on volume/brick stop) posted (#3) for review on master by Samikshan Bairagya (samikshan)

Comment 7 Worker Ant 2017-02-27 22:59:07 UTC
COMMIT: https://review.gluster.org/16689 committed in master by Jeff Darcy (jdarcy) 
------
commit 1e3538baab7abc29ac329c78182b62558da56d98
Author: Samikshan Bairagya <samikshan>
Date:   Mon Feb 20 18:35:01 2017 +0530

    core: Clean up pmap registry up correctly on volume/brick stop
    
    This commit changes the following:
    1. In glusterfs_handle_terminate, send out individual pmap signout
    requests to glusterd for every brick.
    2. Add another parameter to glusterfs_mgmt_pmap_signout function to
    pass the brickname that needs to be removed from the pmap registry.
    3. Make sure pmap_registry_search doesn't break out from the loop
    iterating over the list of bricks per port if the first brick entry
    corresponding to a port is whitespaced out.
    4. Make sure the pmap registry entries are removed for other
    daemons like snapd.
    
    Change-Id: I69949874435b02699e5708dab811777ccb297174
    BUG: 1421590
    Signed-off-by: Samikshan Bairagya <samikshan>
    Reviewed-on: https://review.gluster.org/16689
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Gaurav Yadav <gyadav>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 8 Shyamsundar 2017-05-30 18:42:26 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.11.0, please open a new bug report.

glusterfs-3.11.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-May/000073.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.