Bug 1484225 - [rpc]: EPOLLERR - disconnecting now messages every 3 secs after completing rebalance
Summary: [rpc]: EPOLLERR - disconnecting now messages every 3 secs after completing re...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: rpc
Version: mainline
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Milind Changire
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1483956 1484885
TreeView+ depends on / blocked
 
Reported: 2017-08-23 04:47 UTC by Milind Changire
Modified: 2017-12-08 17:39 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.13.0
Clone Of: 1483956
: 1484885 (view as bug list)
Environment:
Last Closed: 2017-12-08 17:39:07 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Comment 1 Milind Changire 2017-08-23 04:48:18 UTC
Steps to Reproduce:
===================
1. Create 3x2 volume and write data to it
2. Remove brick start to make it 2x2
3. Once rebalance is completed, do commit. 
4. Monitor the glusterd log file

Actual results:
===============
EPOLLERR error message comes every 3 secs.

Comment 2 Worker Ant 2017-08-23 04:49:20 UTC
REVIEW: https://review.gluster.org/18093 (rpc: disable client on disconnection from rebalance) posted (#1) for review on master by Milind Changire (mchangir)

Comment 3 Worker Ant 2017-08-23 10:34:37 UTC
COMMIT: https://review.gluster.org/18093 committed in master by Raghavendra G (rgowdapp) 
------
commit 5b14c11d3cae38bc66006b02217ede485ae30dea
Author: Milind Changire <mchangir>
Date:   Wed Aug 23 10:10:13 2017 +0530

    rpc: disable client on disconnection from rebalance
    
    Problem:
    glusterd rpc code path attempts to reconnect to rebalance process
    via the reconnect timer even after the rebalance process disconnection
    
    Solution:
    Set the clnt->disabled flag to 1 to avoid reconnection and cause
    the clnt object to be unref'd
    
    Change-Id: I4e38eaef45d2fdea86d25e9dff9f1af0cd29cf66
    BUG: 1484225
    Signed-off-by: Milind Changire <mchangir>
    Reviewed-on: https://review.gluster.org/18093
    Smoke: Gluster Build System <jenkins.org>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-by: Raghavendra G <rgowdapp>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 4 Worker Ant 2017-08-24 07:07:02 UTC
REVIEW: https://review.gluster.org/18113 (Revert "rpc: disable client on disconnection from rebalance") posted (#1) for review on master by Milind Changire (mchangir)

Comment 5 Worker Ant 2017-08-24 07:13:12 UTC
REVIEW: https://review.gluster.org/18114 (rpc: disable rpc_clnt_t after relalance process disconnection) posted (#1) for review on master by Milind Changire (mchangir)

Comment 6 Worker Ant 2017-08-24 07:42:04 UTC
REVIEW: https://review.gluster.org/18114 (glusterd: disable rpc_clnt_t after relalance process disconnection) posted (#2) for review on master by Atin Mukherjee (amukherj)

Comment 7 Worker Ant 2017-08-24 13:10:21 UTC
COMMIT: https://review.gluster.org/18114 committed in master by Atin Mukherjee (amukherj) 
------
commit a894d44427649e99d4344a241dc2f9d584a9a691
Author: Milind Changire <mchangir>
Date:   Thu Aug 24 12:39:47 2017 +0530

    glusterd: disable rpc_clnt_t after relalance process disconnection
    
    Problem:
    glusterd continues to connect to rebalance process even after
    the socket connection has disconnected.
    
    Solution:
    rpc_clnt_disable() disables the rpc_clnt_t object and disarms
    all relevant timers and drops refs to the rpc_clnt_t object
    and the transport as well.
    
    Change-Id: I981d6f1cc0087037f1927062c2770a4d5026a619
    BUG: 1484225
    Signed-off-by: Milind Changire <mchangir>
    Reviewed-on: https://review.gluster.org/18114
    Reviewed-by: MOHIT AGRAWAL <moagrawa>
    Tested-by: Atin Mukherjee <amukherj>
    Reviewed-by: Atin Mukherjee <amukherj>
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 8 Worker Ant 2017-08-24 18:47:42 UTC
COMMIT: https://review.gluster.org/18113 committed in master by Atin Mukherjee (amukherj) 
------
commit ea48cae5df118d9b901e7d79cd8726b6f38d65a0
Author: Milind Changire <mchangir>
Date:   Thu Aug 24 12:35:03 2017 +0530

    Revert "rpc: disable client on disconnection from rebalance"
    
    This reverts commit 5b14c11d3cae38bc66006b02217ede485ae30dea.
    
    BUG: 1484225
    Change-Id: I3269d3fc64de3f3cc6f670ea564a87d7725e10fd
    Signed-off-by: Milind Changire <mchangir>
    Reviewed-on: https://review.gluster.org/18113
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Atin Mukherjee <amukherj>
    Tested-by: Atin Mukherjee <amukherj>
    Reviewed-by: Raghavendra G <rgowdapp>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 9 Shyamsundar 2017-12-08 17:39:07 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.13.0, please open a new bug report.

glusterfs-3.13.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-December/000087.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.