Bug 151080 - sftp over a persistent connection (days/weeks) develops a memory leak.
Summary: sftp over a persistent connection (days/weeks) develops a memory leak.
Alias: None
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: openssh   
(Show other bugs)
Version: 3.0
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Tomas Mraz
QA Contact: Brian Brock
Depends On:
Blocks: 156320
TreeView+ depends on / blocked
Reported: 2005-03-14 17:51 UTC by Ken Snider
Modified: 2007-11-30 22:07 UTC (History)
0 users

Fixed In Version: RHSA-2005-550
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2005-09-28 14:31:16 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
strace of the problem (14.10 KB, text/plain)
2005-03-14 17:51 UTC, Ken Snider
no flags Details
Patch (1.50 KB, patch)
2005-05-25 12:01 UTC, Tomas Mraz
no flags Details | Diff

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2005:550 qe-ready SHIPPED_LIVE Low: openssh security update 2005-09-28 04:00:00 UTC

Description Ken Snider 2005-03-14 17:51:37 UTC
Description of problem:
we make use of sftp over a persistent ssh tunnel to move files from one node to
another. We've noticed that, over time, the sftp client connection will grow to
several hundred megabytes in size, meaning we need to restart this service about
weekly to avoid them consuming all available memory.

Version-Release number of selected component (if applicable):

How reproducible:

I've attached an strace of one iteration of the daemon.

Comment 1 Ken Snider 2005-03-14 17:51:37 UTC
Created attachment 111994 [details]
strace of the problem

Comment 2 Tomas Mraz 2005-05-25 12:00:27 UTC
I cannot reproduce this memory leak here. But it's possible that I don't use the
same commands as you do.

Comment 3 Tomas Mraz 2005-05-25 12:01:44 UTC
Created attachment 114822 [details]

Can you try this patch if it helps? It removes some memory leaks on some error

Comment 4 Ken Snider 2005-05-25 12:11:18 UTC
We'll roll this into our server today, it'll be a few days to see if the problem
goes away, of course - there's no easy way to accelerate the "leak rate" on our end.

As for what commands we use, one entire iteration of what the daemon does is in
the strace - that exact command set is what is run, once per minute. The only
thing that changes is the quantity of files received.

Comment 5 Tomas Mraz 2005-05-25 12:34:59 UTC
I can see the commands in the strace, however the command lines aren't complete
because strace cuts them. And there seem to be many failed commands in the
strace which I don't suppose to be the case in your real environment.
The patch is backported from 4.0p1 and there are only fixes for few error paths
in the code so I wouldn't be very surprised if it didn't help.

Comment 6 Ken Snider 2005-05-25 12:45:57 UTC
Actually, it's a capture from our production environment. :) a lot of distinct
files are inserted here, so there are a lot of gets that simply have no file
(because of the directory size, it's not feasible to stat the directory, the
directory contents can be more than a megabyte when transferred, even though
it's rotated every 60 seconds or so).

So, it's quite possible that the leaks *are* occuring only on errors. Either
way, we should know definitively after I've given the patch a few days to run.

Comment 7 Tomas Mraz 2005-06-06 20:23:13 UTC
So did the patch help?

Comment 8 Ken Snider 2005-06-06 21:47:54 UTC
Actually yes! Significantly! The memory footprints of the instances are now
within a meg of each other (most within a few K). Compare this to several
hundred Megabytes of discrepancy prior to this patch, and it's obviously a
significant improvement.

I can't say for certain that *all* leaks have been caught, but certainly 99% of
them were covered in the patch.

Comment 12 Red Hat Bugzilla 2005-09-28 14:31:16 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.