Bug 434294 - lftp corrupts data when using (m)put's -c option on sftp transport
lftp corrupts data when using (m)put's -c option on sftp transport
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: lftp (Show other bugs)
x86_64 Linux
low Severity high
: rc
: ---
Assigned To: Jiri Skala
: Patch
Depends On:
  Show dependency treegraph
Reported: 2008-02-22 10:56 EST by Peter K
Modified: 2014-11-09 17:30 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-09-02 05:38:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Patch for lftp (1.43 KB, patch)
2008-05-02 15:44 EDT, Martin Nagy
no flags Details | Diff

  None (edit)
Description Peter K 2008-02-22 10:56:26 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20080201 Firefox/

Description of problem:
When using sftp transport with lftp (like lftp sftp://user@host.domain/) files are not allways written correctly if you use the -c option to one of put, mput or reput. The mirror subcommand of lftp may also be affected by has not been investigated.

Files on the remote side are typically created with either size==0 or as large sparse files (often 4gig but sometimes alot larger like 320gig or even more). Files can also be created and transfered with correct size but bad content. Lftp does not typically report an error so this is silent data corruption.

All tested versions of lftp have had the same problem not just the shipped el5.1 version. I tested with up to 3.6.3.

The problem has been very easy to reproduce but not consistent. I have managed to mput quite a few files and get all of them right but for most of the time 50%+ of the files are broken as described above.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. on host 1, I had 8 files of different sizes at a total of ~200 megs but I don't think it matters much. Assuming they are called files.* and are in the current dir, create a md5sum file, "md5sum files.* > MD5SUMS"
2. on host1, lftp sftp://host2
3. with suitable remote dir do "mput -c files.*"
4. on host2, verify the md5sums, md5sum -c MD5SUMS (yes copy the MD5SUMS file over with for example scp). If you have huge sparse files you may want to delete them before running md5sum...

Actual Results:
ls -l on the remote side shows a mix of zero sized files, huge sparse files and correctly transfered files. md5sum shows corrupted files.

Expected Results:
all files to be tranfered correctly with no corruption _OR_ that lftp would refuse the -c option with sftp transport. (Our local quick fix is to remove the -c options)

Additional info:
Here I had a look at what I got on the remote side after the transfer...
[cap@crux test]$ ls -l
total 223920
-rw-r--r-- 1 cap nsc 8391735724893090349 Jul 20  2007 files.1
-rw-r--r-- 1 cap nsc              174368 Oct 15 11:47 files.2
-rw-r--r-- 1 cap nsc              464188 Jul 25  2007 files.3
-rw-r--r-- 1 cap nsc              199498 Jan 25  2007 files.4
-rw-r--r-- 1 cap nsc             3035523 Jul 20  2007 files.5
-rw-r--r-- 1 cap nsc           217583887 Apr 30  2006 files.6
-rw-r--r-- 1 cap nsc             7604980 Jan 29 09:17 files.7
-rw-r--r-- 1 cap nsc              212636 Feb 11 15:29 files.8
-rw-r--r-- 1 cap nsc                 486 Feb 22 16:45 MD5SUMS
[cap@crux test]$ rm files.1
[cap@crux test]$ md5sum -c MD5SUMS
md5sum: files.1: No such file or directory
files.1: FAILED open or read
files.2: OK
files.3: OK
files.4: FAILED
files.5: FAILED
files.6: FAILED
files.7: FAILED
files.8: FAILED
md5sum: WARNING: 1 of 8 listed files could not be read
md5sum: WARNING: 5 of 7 computed checksums did NOT match
[cap@crux test]$
Comment 1 Martin Nagy 2008-05-02 10:13:29 EDT
I can confirm to be able to reproduce this. It seems to have been fixed in the
latest 3.7.1 release.
Comment 2 Martin Nagy 2008-05-02 15:44:37 EDT
Created attachment 304405 [details]
Patch for lftp

This is a backported patch from upstream, please try. This has solved the issue
for me.
Comment 4 RHEL Product and Program Management 2008-06-04 18:46:28 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
Comment 13 errata-xmlrpc 2009-09-02 05:38:12 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.