Red Hat Bugzilla – Bug 434294
lftp corrupts data when using (m)put's -c option on sftp transport
Last modified: 2014-11-09 17:30:51 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:18.104.22.168) Gecko/20080201 Firefox/22.214.171.124
Description of problem:
When using sftp transport with lftp (like lftp sftp://firstname.lastname@example.org/) files are not allways written correctly if you use the -c option to one of put, mput or reput. The mirror subcommand of lftp may also be affected by has not been investigated.
Files on the remote side are typically created with either size==0 or as large sparse files (often 4gig but sometimes alot larger like 320gig or even more). Files can also be created and transfered with correct size but bad content. Lftp does not typically report an error so this is silent data corruption.
All tested versions of lftp have had the same problem not just the shipped el5.1 version. I tested with up to 3.6.3.
The problem has been very easy to reproduce but not consistent. I have managed to mput quite a few files and get all of them right but for most of the time 50%+ of the files are broken as described above.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. on host 1, I had 8 files of different sizes at a total of ~200 megs but I don't think it matters much. Assuming they are called files.* and are in the current dir, create a md5sum file, "md5sum files.* > MD5SUMS"
2. on host1, lftp sftp://host2
3. with suitable remote dir do "mput -c files.*"
4. on host2, verify the md5sums, md5sum -c MD5SUMS (yes copy the MD5SUMS file over with for example scp). If you have huge sparse files you may want to delete them before running md5sum...
ls -l on the remote side shows a mix of zero sized files, huge sparse files and correctly transfered files. md5sum shows corrupted files.
all files to be tranfered correctly with no corruption _OR_ that lftp would refuse the -c option with sftp transport. (Our local quick fix is to remove the -c options)
Here I had a look at what I got on the remote side after the transfer...
[cap@crux test]$ ls -l
-rw-r--r-- 1 cap nsc 8391735724893090349 Jul 20 2007 files.1
-rw-r--r-- 1 cap nsc 174368 Oct 15 11:47 files.2
-rw-r--r-- 1 cap nsc 464188 Jul 25 2007 files.3
-rw-r--r-- 1 cap nsc 199498 Jan 25 2007 files.4
-rw-r--r-- 1 cap nsc 3035523 Jul 20 2007 files.5
-rw-r--r-- 1 cap nsc 217583887 Apr 30 2006 files.6
-rw-r--r-- 1 cap nsc 7604980 Jan 29 09:17 files.7
-rw-r--r-- 1 cap nsc 212636 Feb 11 15:29 files.8
-rw-r--r-- 1 cap nsc 486 Feb 22 16:45 MD5SUMS
[cap@crux test]$ rm files.1
[cap@crux test]$ md5sum -c MD5SUMS
md5sum: files.1: No such file or directory
files.1: FAILED open or read
md5sum: WARNING: 1 of 8 listed files could not be read
md5sum: WARNING: 5 of 7 computed checksums did NOT match
I can confirm to be able to reproduce this. It seems to have been fixed in the
latest 3.7.1 release.
Created attachment 304405 [details]
Patch for lftp
This is a backported patch from upstream, please try. This has solved the issue
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release. Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products. This request is not yet committed for inclusion in an Update
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.