Bug 1097224

Summary: Disconnects of peer and brick is logged while snapshot creations were in progress during IO
Product: [Community] GlusterFS Reporter: Vijaikumar Mallikarjuna <vmallika>
Component: glusterdAssignee: krishnan parthasarathi <kparthas>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.5.0CC: bugs, gluster-bugs, nlevinki, nsathyan, rhinduja, smohan, ssamanta, vbellur
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.6.0beta1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1096729 Environment:
Last Closed: 2014-11-11 08:32:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1096729, 1098021, 1098025, 1104459, 1104462    
Bug Blocks:    

Comment 1 Anand Avati 2014-05-13 12:19:57 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#1) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 2 Anand Avati 2014-05-14 03:35:43 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#2) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 3 Anand Avati 2014-05-14 05:18:31 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#3) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 4 Anand Avati 2014-05-14 05:23:02 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#4) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 5 Anand Avati 2014-05-14 05:32:02 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#5) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 6 Anand Avati 2014-05-14 05:48:55 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#6) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 7 Anand Avati 2014-05-14 06:46:54 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#7) for review on master by Krishnan Parthasarathi (kparthas)

Comment 8 Anand Avati 2014-05-14 07:32:51 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#8) for review on master by Krishnan Parthasarathi (kparthas)

Comment 9 Anand Avati 2014-05-14 10:39:07 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#9) for review on master by Krishnan Parthasarathi (kparthas)

Comment 10 Anand Avati 2014-05-15 14:00:18 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#10) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 11 Anand Avati 2014-05-15 14:16:16 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#11) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 12 Anand Avati 2014-05-16 09:02:16 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#12) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 13 Anand Avati 2014-05-16 09:32:34 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#13) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 14 Anand Avati 2014-05-19 07:19:18 UTC
REVIEW: http://review.gluster.org/7753 (glusterd: Disable ping-timer between glusterd and brick process) posted (#14) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 15 Anand Avati 2014-05-20 02:04:02 UTC
COMMIT: http://review.gluster.org/7753 committed in master by Vijay Bellur (vbellur) 
------
commit c13c1fd5a487eeca269e8b62114f607487ee30f6
Author: Vijaikumar M <vmallika>
Date:   Tue May 13 17:36:01 2014 +0530

    glusterd: Disable ping-timer between glusterd and brick process
    
    When there are too many IO happening, brick process epoll thread
    will be busy and fails to respond to the glusterd pick packet within
    30sec.
    Also epoll thread can be blocked by a big-lock.
    
    Solution is to disable ping-timer by default and only enable where ever
    required
    
    Later when the epoll thread model changed and made lighter,
    we need to revert back this change. http://review.gluster.com/3842 is
    one such approach.
    
    Change-Id: I7f80ad3eb00f7d9c4d4527305932f7cf4920e73f
    BUG: 1097224
    Signed-off-by: Vijaikumar M <vmallika>
    Reviewed-on: http://review.gluster.org/7753
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 16 Anand Avati 2014-06-05 08:30:37 UTC
REVIEW: http://review.gluster.org/7989 (gluster: Set ping-timeout between glusterd peer rpc connection) posted (#2) for review on master by Vijaikumar Mallikarjuna (vmallika)

Comment 17 Anand Avati 2014-06-07 03:52:19 UTC
COMMIT: http://review.gluster.org/7989 committed in master by Krishnan Parthasarathi (kparthas) 
------
commit d64d0129ee8e0c32f027cf9df17e8debfaf73ab2
Author: Vijaikumar M <vmallika>
Date:   Thu Jun 5 13:26:22 2014 +0530

    gluster: Set ping-timeout between glusterd peer rpc connection
    
    With Patch# http://review.gluster.org/#/c/7753/, ping-timeout
    got disabled between glusterd peer rpc connection.
    This would create regression for bug#1034479
    
    Change-Id: Ib7a756fce7d349ce5b7b4c6a287f2d040c83fc76
    BUG: 1097224
    Signed-off-by: Vijaikumar M <vmallika>
    Reviewed-on: http://review.gluster.org/7989
    Reviewed-by: Rajesh Joseph <rjoseph>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Tested-by: Krishnan Parthasarathi <kparthas>

Comment 18 Niels de Vos 2014-09-22 12:40:19 UTC
A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 19 Niels de Vos 2014-11-11 08:32:27 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users