Bug 208155 - crash on syncing large directory trees
crash on syncing large directory trees
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: rsync (Show other bugs)
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Jan Zeleny
Depends On:
  Show dependency treegraph
Reported: 2006-09-26 13:40 EDT by David Cantrell
Modified: 2010-03-05 04:46 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2010-03-05 04:46:09 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description David Cantrell 2006-09-26 13:40:44 EDT
rsync on RHEL4 (version 2.6.3, protocol 28) crashes when syncing large directory
trees over ssh.  The command I use is:

rsync -Pav --delete --progress /source/path/ user@remote:/dest/path/

The sync will start to run and crash after a while.  The message given is phase
"unknown" [sender]: connection reset by peer.  The tree I was trying to sync was
1198MB in size with 1046 directories and 25516 files.

It seems related to this problem:

I've been able to work around the problem a bit playing with the --bwlimit and
--timeout options, but those just seem to delay the crash.  I have not really
looked in to it beyond searching for an existing bug.
Comment 1 Matt Whitted 2007-01-02 09:45:48 EST
We are experiencing similar problems -- it looks like this bug was opened over 3
months ago and has no resolution yet.  Has this been investigated?  
Comment 2 David Cantrell 2007-03-06 14:02:26 EST
Would like a fix for this to show up in a RHEL4 update.  Setting flags.
Comment 3 Simo Sorce 2007-05-14 15:27:18 EDT
Was this between 2 RHEL4 servers?
Or different machines?
Can you reproduce this at will? And if so how do you reproduce it?
Comment 4 David Cantrell 2007-05-14 15:38:44 EDT
Sort of.  It was between a CentOS 4.4 server and a RHEL 4.4 server.  I can
reproduce it using the description given in the first comment.  It's pretty
simple to reproduce.
Comment 5 Nalin Dahyabhai 2007-09-14 14:53:03 EDT
Checked with David, and it looks like this was happening between 2.6.3 on both
ends, and isn't reproducible when both ends are running 2.6.9.
Comment 6 Simo Sorce 2007-09-24 12:00:04 EDT
I can't reproduce this with 1-2G data sets on RHEL4 <-> RHEL4
If someone has more info on how to reliably reproduce it, it will make a lot
more easier to find what's wrong and patch it.

Comment 7 Simo Sorce 2007-10-05 15:34:22 EDT
It seem that David can't reproduce the bug as he changed the test environment
and I can't reproduce it as well.
Can you Matt provide a reproducible test case?
It would really help a lot to find out the exact conditions to reproduce it so
that I can actually fix it.

Comment 8 Matt Whitted 2007-10-05 16:16:43 EDT
We worked around the issue by changing our iptables rulesets.  Originally we
used connection tracking to protect the system.  When we removed any rules that
use connection tracking, we stopped experiencing the issues reported here.

As such, I cannot tell if this is rsync related or not, but I would recommend
setting up connection tracking rules something roughly as follows to see if this
reproduces the problem:

iptables -F
iptables -I INPUT 1 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -I INPUT 2 -j DROP
iptables -I OUTPUT 1 -m state --state NEW -j ACCEPT

Given our environment, with the above, the following rough command would fail
after several minutes given gigabit ethernet between hosts:

rsync -avr --delete --progress --stats user@remote:/data /data
Comment 9 RHEL Product and Program Management 2008-02-01 14:12:06 EST
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 10 Jan Zeleny 2010-03-05 04:46:09 EST
Since the workaround implies that this issue might not be rsync related, I'm closing this bug.

Note You need to log in before you can comment on or make changes to this bug.