Bug 1420327 - CURL 7.29 cannot connect to FTPS using proxytunnel
Summary: CURL 7.29 cannot connect to FTPS using proxytunnel
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: curl
Version: 7.3
Hardware: All
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Kamil Dudka
QA Contact: Karel Srot
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-08 13:14 UTC by dbodnarc
Modified: 2020-06-11 13:16 UTC (History)
2 users (show)

Fixed In Version: curl-7.29.0-42.el7
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-08-01 17:02:31 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:2016 0 normal SHIPPED_LIVE Moderate: curl security, bug fix, and enhancement update 2017-08-01 18:02:02 UTC

Description dbodnarc 2017-02-08 13:14:04 UTC
Description of problem:
Our customer is using curl (on RHEL6) to work with remote FTPS server through proxy (using proxytunnel). After he tried curl@RHEL7, he found out that the mentioned configuration doesn't work anymore

> CURL doesn't work with remote FTPS server through proxy (with --proxytunnel) on RHEL7

Version-Release number of selected component (if applicable):
[root@fastvm-r7-3-73 ~]# curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.21 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets 
[root@fastvm-r7-3-73 ~]# rpm -q curl nss libcurl
curl-7.29.0-35.el7.x86_64
nss-3.21.3-2.el7_3.x86_64
libcurl-7.29.0-35.el7.x86_64

How reproducible:
curl -v --user "anonymous:ftp@example.com" ftps://192.168.22.73/pub/test.txt -k -x 192.168.22.72:3128 -proxytunnel

 192.168.22.72 - test squid proxy 
 192.168.22.73 - test vsftp server (self-signed certificate)


Steps to Reproduce:
$ curl -v --user "anonymous:ftp@example.com" ftps://192.168.22.73/pub/test.txt -k -x 192.168.22.72:3128 -proxytunnel
 ^^ doesn't work

Actual results:
* About to connect() to proxy 192.168.22.72 port 3128 (#0)
*   Trying 192.168.22.72...
* Connected to 192.168.22.72 (192.168.22.72) port 3128 (#0)
* Establish HTTP proxy tunnel to 192.168.22.73:990
* Server auth using Basic with user 'anonymous'
> CONNECT 192.168.22.73:990 HTTP/1.1
> Host: 192.168.22.73:990
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* Closing connection 0
curl: (81) Socket not ready for send/recv



Expected results:
* About to connect() to proxy 192.168.22.72 port 3128 (#0)
*   Trying 192.168.22.72... connected
* Connected to 192.168.22.72 (192.168.22.72) port 3128 (#0)
* Establish HTTP proxy tunnel to 192.168.22.73:990
* Server auth using Basic with user 'anonymous'
> CONNECT 192.168.22.73:990 HTTP/1.1
> Host: 192.168.22.73:990
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* Unable to initialize NSS database
* Initializing NSS with certpath: none
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: CN=test.example.com,O=Default Company Ltd,L=Default City,C=XX
* 	start date: Feb 07 08:41:55 2017 GMT
* 	expire date: Feb 07 08:41:55 2018 GMT
* 	common name: test.example.com
* 	issuer: CN=test.example.com,O=Default Company Ltd,L=Default City,C=XX
< 220 (vsFTPd 3.0.2)
> USER anonymous
< 331 Please specify the password.
> PASS ftp@example.com
< 230 Login successful.
> PBSZ 0
< 200 PBSZ set to 0.
> PROT P
< 200 PROT now Private.
> PWD
< 257 "/"
* Entry path is '/'
> CWD pub
< 250 Directory successfully changed.
> EPSV
* Connect data stream passively
< 229 Entering Extended Passive Mode (|||16622|).
*   Trying 192.168.22.72... connected
* Connecting to 192.168.22.73 (192.168.22.72) port 3128
* Establish HTTP proxy tunnel to 192.168.22.73:16622
* Server auth using Basic with user 'anonymous'
> CONNECT 192.168.22.73:16622 HTTP/1.1
> Host: 192.168.22.73:16622
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
> TYPE I
< 200 Switching to Binary mode.
> SIZE test.txt
< 213 8
> RETR test.txt
< 150 Opening BINARY mode data connection for test.txt (8 bytes).
* Doing the SSL/TLS handshake on the data stream
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: CN=test.example.com,O=Default Company Ltd,L=Default City,C=XX
* 	start date: Feb 07 08:41:55 2017 GMT
* 	expire date: Feb 07 08:41:55 2018 GMT
* 	common name: test.example.com
* 	issuer: CN=test.example.com,O=Default Company Ltd,L=Default City,C=XX
* Maxdownload = -1
* Getting file with size: 8
testftp
* Remembering we are in dir "pub/"
< 226 Transfer complete.
* Connection #0 to host 192.168.22.72 left intact
> QUIT
< 221 Goodbye.
* Closing connection #0


Additional info:
Issue is reproducible using curl7.29.0-25/curl7.29.0-35 on RHEL7

At the same time this configuration works well using CURL version shipped with RHEL6:
[root@fastvm-r6-8-68 ~]# curl -V
curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz 
[root@fastvm-r6-8-68 ~]# rpm -q curl nss libcurl
curl-7.19.7-52.el6.x86_64
nss-3.21.0-8.el6.x86_64
libcurl-7.19.7-52.el6.x86_64

Comment 3 Kamil Dudka 2017-02-08 16:25:47 UTC
Thank you for reporting the bug!  I am able to reproduce it locally and will find the cause.

Comment 4 Kamil Dudka 2017-02-09 09:31:10 UTC
This seems to be broken by the following upstream commit:

https://github.com/curl/curl/commit/curl-7_29_0~91

We need to fix the code such that the CONNECT phase blocks.

Comment 6 Kamil Dudka 2017-02-09 16:46:27 UTC
I have pushed a patch upstream to fix this bug:

https://github.com/curl/curl/commit/curl-7_52_1-124-g8fa5409

It should be easy and safe to backport for RHEL-7 curl.

Comment 15 Kamil Dudka 2017-03-27 13:54:47 UTC
Thank you for providing the additional info!  I am able to reproduce it locally and I will fix it by applying the following upstream commits:

https://github.com/curl/curl/commit/curl-7_37_1-19-ga4cece3
https://github.com/curl/curl/commit/curl-7_43_0-4-gb88f980

Note that the above commits fix the logic at HTTP level while the original fix for this bug fixes the TLS backend of libcurl.

Comment 18 Kamil Dudka 2017-03-28 11:59:07 UTC
Could you please verify that curl-7.29.0-40.el7 works as expected?

Comment 19 Kamil Dudka 2017-03-28 12:08:47 UTC
Ooops, still not perfect.  It seems to hang in certain situations.  This will need some additional debugging...

Comment 20 Kamil Dudka 2017-03-28 15:30:17 UTC
Additional commit pushed upstream:

https://github.com/curl/curl/commit/curl-7_53_1-120-g2549831

Still debugging...

Comment 21 Kamil Dudka 2017-03-29 10:56:12 UTC
One more commit needs to be picked from upstream:

https://github.com/curl/curl/commit/curl-7_33_0-60-gd44b014

Comment 25 Kamil Dudka 2017-03-29 14:19:10 UTC
Could you please verify that curl-7.29.0-42.el7 works as expected?

Comment 26 dbodnarc 2017-03-31 10:54:44 UTC
I've checked the latest curl-7.29.0-42.el7 and it works fine for me now
thank you.

Comment 28 Kamil Dudka 2017-03-31 11:33:18 UTC
Perfect.  Thanks for confirmation!

Comment 29 Karel Srot 2017-05-23 11:48:22 UTC
Just FYI, when following the reproducer I have encountered that when using -proxytunnel parameter curl seem to work but it is complaining with:

Warning: Invalid character is found in given range. A specified range MUST 
Warning: have only digits in 'start'-'stop'. The server's response to this 
Warning: request is uncertain.

The right approach is to specify the option as --proxytunnel.

Comment 30 Kamil Dudka 2017-05-23 12:19:23 UTC
(In reply to Karel Srot from comment #29)
> Just FYI, when following the reproducer I have encountered that when using
> -proxytunnel parameter curl seem to work but it is complaining with:
> 
> Warning: Invalid character is found in given range. A specified range MUST 
> Warning: have only digits in 'start'-'stop'. The server's response to this 
> Warning: request is uncertain.

This was caused by "-proxytunnel" being interpreted as URL containing a glob.  The glob parser can be disabled by the --globoff (-g) option of curl.

Comment 32 errata-xmlrpc 2017-08-01 17:02:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2016


Note You need to log in before you can comment on or make changes to this bug.