Bug 167398

Summary: connections don't expire when ipvs receives lots of incoming ICMPs
Product: Red Hat Enterprise Linux 4 Reporter: Justin Albstmeijer <justin>
Component: kernelAssignee: Jeff Burke <jburke>
Status: CLOSED ERRATA QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.0CC: akrherz, aveseb, davem, jbaron, nhorman, tgraf
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
URL: http://archive.linuxvirtualserver.org/html/lvs-users/2005-05/msg00002.html
Whiteboard:
Fixed In Version: RHBA-2007-0304 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-05-01 23:21:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Justin Albstmeijer 2005-09-02 11:17:43 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050720 Fedora/1.0.6-1.1.fc4 Firefox/1.0.6

Description of problem:
Using ipvs to loadbalance caching dns servers results in memory usage problems on the directors.
Because ipvs connections are not expired the ip_vs_conn slab keeps growing and growing untill the box runs out of memory.


Version-Release number of selected component (if applicable):
kernel-2.6.9-11

How reproducible:
Always

Steps to Reproduce:
1. configure an ipvs director to loadbalance a couple of cachingnameservers
2. have loads of clients use this loadbalanced environment
3. watch the memory usage and open udp connections grow till the machine OOM's
  

Actual Results:  machine OOM's

Expected Results:  machine should clean up old connections

Additional info:


The following patch, fixes the problem.
This fix has been included to 2.6.12-rc5.

-------------------------------------------------------------------------
ChangeLog-2.6.12-rc5

Julian Anastasov <ja>:
    [IP_VS]: Remove extra __ip_vs_conn_put() for incoming ICMP.

    Remove extra __ip_vs_conn_put for incoming ICMP in direct routing
    mode. Mark de Vries reports that IPVS connections are not leaked anymore.

    Signed-off-by: Julian Anastasov <ja>
    Signed-off-by: David S. Miller <davem>


---------------------------------------------------------------------------
diff -ur v2.6.12-rc4/linux/net/ipv4/ipvs/ip_vs_xmit.c linux/net/ipv4/ipvs/ip_vs_xmit.c
--- v2.6.12-rc4/linux/net/ipv4/ipvs/ip_vs_xmit.c	2004-08-31 08:09:31.000000000 +0300
+++ linux/net/ipv4/ipvs/ip_vs_xmit.c	2005-05-09 00:31:47.810807232 +0300
@@ -520,7 +520,6 @@
 			rc = NF_ACCEPT;
 		/* do not touch skb anymore */
 		atomic_inc(&cp->in_pkts);
-		__ip_vs_conn_put(cp);
 		goto out;
 	}

Comment 2 Lon Hohberger 2007-02-01 15:06:25 UTC
*** Bug 220149 has been marked as a duplicate of this bug. ***

Comment 3 Lon Hohberger 2007-02-01 15:14:38 UTC
There's a bigger patch attached to #220149, but I don't know the implications of
the other parts of the patch.  Link:

https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=145159

Comment 4 Lon Hohberger 2007-02-01 15:21:43 UTC
Additional information provided by reporter of #220149:

http://marc.theaimsgroup.com/?l=linux-virtual-server&m=111494344303632&w=2


Comment 7 Lon Hohberger 2007-02-01 18:07:17 UTC
*** Bug 176939 has been marked as a duplicate of this bug. ***

Comment 8 Neil Horman 2007-02-01 19:26:39 UTC
I think we can ignore the extra portions of the larger patch.  The patch
targeted specifically to this problem is identical to this upstream commit:
 d9fa0f392b20b2b8e3df379c44194492a2446c6e
Jeff I say you go ahead and post a backport of that commit.  Its been tested by
the reporter and its been upstream for some time.  lets not go fixing more than
the probelm described by the bug.

Comment 10 RHEL Program Management 2007-02-02 05:04:14 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 12 Jeff Burke 2007-02-03 13:20:43 UTC
Patch posted to internal mailing list for review, Awaiting kernel developer ACKs.

Comment 15 Jason Baron 2007-02-13 15:20:32 UTC
committed in stream U5 build 47. A test kernel with this patch is available from
http://people.redhat.com/~jbaron/rhel4/


Comment 16 Jay Turner 2007-02-13 15:59:05 UTC
QE ack for RHEL4.5.

Comment 19 Red Hat Bugzilla 2007-05-01 23:21:01 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2007-0304.html