Bug 1254607

Summary:	rpc: Address issues with transport object reference and leak
Product:	[Community] GlusterFS	Reporter:	Soumya Koduri <skoduri>
Component:	rpc	Assignee:	Soumya Koduri <skoduri>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.7.3	CC:	bugs, gluster-bugs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.7.4	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1250797	Environment:
Last Closed:	2015-09-09 09:40:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1250797
Bug Blocks:

Description Soumya Koduri 2015-08-18 13:44:11 UTC

+++ This bug was initially created as a clone of Bug #1250797 +++

Description of problem:

We have seen a couple of issues with transport object reference and leak in rpc
layer, particularly with 'rpc_clnt_disable', 

a) Once the transport object is disassociated from rpc->conn, it should not be 
referenced further. Fix is needed in 'rpc_clnt_remove_ping_timer_locked'.

b) Unref transport object to avoid memory leak.

--- Additional comment from Anand Avati on 2015-08-08 06:50:48 EDT ---

COMMIT: http://review.gluster.org/11845 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit 6e8c137fb7deca75817d617d40ebd58bbcae186a
Author: Soumya Koduri <skoduri>
Date:   Thu Aug 6 06:22:38 2015 +0530

    rpc-clnt: Check for transport object during rpc connection cleanup
    
    While doing glfs_fini(), all the xlators are first notified of
    PARENT_DOWN. protocol-client xlator on receving that notification
    does rpc_clnt_disable which disassociates rpc->conn with its
    transport object and does socket shutdown. So any further references
    to conn->trans should not happen during rpc connection cleanup which
    is done mainly as part of epoll event handling of EPOLLERR/EPOLLHUP.
    
    BUG: 1250797
    Change-Id: I7d0346451d40372ad59a6a2326176fbb1e694712
    Signed-off-by: Soumya Koduri <skoduri>
    Reviewed-on: http://review.gluster.org/11845
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>

Comment 1 Anand Avati 2015-08-19 03:38:14 UTC

REVIEW: http://review.gluster.org/11953 (rpc-clnt: Check for transport object during rpc connection cleanup) posted (#2) for review on release-3.7 by Kaleb KEITHLEY (kkeithle)

Comment 2 Anand Avati 2015-08-19 11:11:13 UTC

COMMIT: http://review.gluster.org/11953 committed in release-3.7 by Kaleb KEITHLEY (kkeithle) 
------
commit 4ec99ad55f2ad74d67591cda68320c26235a9e12
Author: Soumya Koduri <skoduri>
Date:   Thu Aug 6 06:22:38 2015 +0530

    rpc-clnt: Check for transport object during rpc connection cleanup
    
    While doing glfs_fini(), all the xlators are first notified of
    PARENT_DOWN. protocol-client xlator on receving that notification
    does rpc_clnt_disable which disassociates rpc->conn with its
    transport object and does socket shutdown. So any further references
    to conn->trans should not happen during rpc connection cleanup which
    is done mainly as part of epoll event handling of EPOLLERR/EPOLLHUP.
    
    This is a backport of the below fix-
     http://review.gluster.org/#/c/11845/
    
    BUG: 1254607
    Change-Id: I619ec00fd061f77c9b04dfa6fd139620cb44189b
    Signed-off-by: Soumya Koduri <skoduri>
    Reviewed-on: http://review.gluster.org/11845
    Reviewed-by: Raghavendra G <rgowdapp>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-on: http://review.gluster.org/11953
    Tested-by: NetBSD Build System <jenkins.org>

Comment 3 Kaushal 2015-09-09 09:40:08 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.4, please open a new bug report.

glusterfs-3.7.4 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/12496
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user