Bug 879592 - cannot upload large file with curl over scp
Summary: cannot upload large file with curl over scp
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: curl
Version: 6.3
Hardware: i386
OS: Unspecified
high
medium
Target Milestone: rc
: ---
Assignee: Kamil Dudka
QA Contact: Karel Srot
URL:
Whiteboard:
Depends On:
Blocks: 881827
TreeView+ depends on / blocked
 
Reported: 2012-11-23 12:04 UTC by Karel Srot
Modified: 2013-08-05 02:02 UTC (History)
3 users (show)

Fixed In Version: curl-7.19.7-33.el6
Doc Type: Bug Fix
Doc Text:
Cause: libcurl used the old libssh2 API for uploading files over SCP, which limited the maximum size of files being transferred on 32bit architectures. Consequence: The 32bit packages of libcurl were not able to transfer large files over SCP. Fix: The code of libcurl has been changed to use the new libssh2 API for SCP upload, which does not suffer from such a limitation. Result: libcurl can now upload large files over SCP.
Clone Of:
Environment:
Last Closed: 2013-02-21 10:09:49 UTC
Target Upstream Version:


Attachments (Terms of Use)
backport of the above mentioned upstream commits (4.54 KB, patch)
2012-11-23 13:25 UTC, Kamil Dudka
ovasik: review+
Details | Diff


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:0393 normal SHIPPED_LIVE curl bug fix and enhancement update 2013-02-20 20:51:32 UTC

Description Karel Srot 2012-11-23 12:04:22 UTC
Description of problem:

Found an issue with uploading large file via scp on i386. After 1GB of data is transmitted the connection somehow stucks and the ammount of transferred data doesn't increase anymore. 

looks like this:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0 1026M      0  6443k --:--:--  0:02:43 --:--:--     0

It is not a regression, same issue happens with libssh2, curl, libcurl from rhel6.3.

Version-Release number of selected component (if applicable):
libssh2-1.4.2-1.el6.i686
libcurl-7.19.7-32.el6.i686
curl-7.19.7-32.el6.i686


How reproducible:
always

Steps to Reproduce:
1.  prepare huge file, ~2GB or so
2.  prepare test user and ssh keys
3.
# curl -T ~curltestuser/hugefile -u curltestuser: --key /root/.ssh/id_rsa --pubkey /root/.ssh/id_rsa.pub scp://localhost/home/curltestuser/uploadedfile
  
Actual results:
around 1GB of data is transfered and from that moment the destination file doesn't grow anymore

# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 57835520 Nov 23 11:53 
/home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 81035264 Nov 23 11:53 
/home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 106610688 Nov 23 11:53 /home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 138182656 Nov 23 11:53 /home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 154959872 Nov 23 11:53 /home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 182255616 Nov 23 11:53 /home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 1073741824 Nov 23 11:54 /home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 1073741824 Nov 23 11:54 /home/curltestuser/uploadedfile
# ls -l ~curltestuser/uploadedfile 
-rw-r--r--. 1 curltestuser curltestuser 1073741824 Nov 23 11:54 /home/curltestuser/uploadedfile


Expected results:
whole file is transferred

Additional info:

Comment 1 Karel Srot 2012-11-23 12:06:33 UTC
forgot to mention that I could reproduce the issue on i386, other architectures transferred the file properly.

Comment 3 Kamil Dudka 2012-11-23 13:25:51 UTC
Created attachment 650441 [details]
backport of the above mentioned upstream commits

Comment 7 RHEL Program Management 2012-11-26 08:00:41 UTC
This request was evaluated by Red Hat Product Management for
inclusion in a Red Hat Enterprise Linux release.  Product
Management has requested further review of this request by
Red Hat Engineering, for potential inclusion in a Red Hat
Enterprise Linux release for currently deployed products.
This request is not yet committed for inclusion in a release.

Comment 12 errata-xmlrpc 2013-02-21 10:09:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0393.html


Note You need to log in before you can comment on or make changes to this bug.