Bug 1248829

Hm.  It appears to be a problem with the nss library.

(Oh, and this happens in Fedora 21, and until I solve it for Fedora 22, I can't do much in Fedora 22.)

$ rpm -q nss-devel openssl-devel
nss-devel-3.19.2-1.0.fc21.x86_64
openssl-devel-1.0.1k-11.fc21.x86_64

If I build curl with nss, I get the same behavior, but if I build it with openssl, it works.

./configure --prefix=$HOME/local/curl_install --with-openssl

$ curl --version
curl 7.43.0 (x86_64-unknown-linux-gnu) libcurl/7.43.0 OpenSSL/1.0.1k zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets 
$ curl -v 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-f21&arch=x86_64'
*   Trying <proxyip>...
* Connected to <proxyip> (<proxyip>) port 8080 (#0)
* Establish HTTP proxy tunnel to mirrors.fedoraproject.org:443
* Proxy auth using Basic with user 'cogljj'
> CONNECT mirrors.fedoraproject.org:443 HTTP/1.1
> Host: mirrors.fedoraproject.org:443
> Proxy-Authorization: Basic <data>
> User-Agent: curl/7.43.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* NPN, no overlap, use HTTP1.1
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Unknown (67):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
* 	 subject: C=US; ST=North Carolina; L=Raleigh; O=Red Hat Inc.; CN=*.fedoraproject.org
* 	 start date: 2014-04-22 00:00:00 GMT
* 	 expire date: 2017-04-26 12:00:00 GMT
* 	 subjectAltName: mirrors.fedoraproject.org matched
* 	 issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
* 	 SSL certificate verify ok.
> GET /metalink?repo=updates-released-f21&arch=x86_64 HTTP/1.1
> Host: mirrors.fedoraproject.org
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Fri, 31 Jul 2015 19:59:47 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
< Content-Length: 10039
< AppTime: D=261752
< AppServer: proxy03.fedoraproject.org
< Content-Type: application/metalink+xml
< 
<?xml version="1.0" encoding="utf-8"?>

Ok, this looks like a curl issue -> reassigning.

(In reply to Josh Cogliati from comment #0)
> The problem seems to be that curl is using a http proxy tunnel for some
> reason.

The reason is that you asked curl to use TLS, so curl will not send possibly sensitive data to the proxy in plain text:

https://github.com/bagder/curl/blob/bb0acba6/lib/url.c#L5584

You need a proxy server that supports CONNECT to let curl speak HTTPS over it.

(In reply to Kamil Dudka from comment #3)
> (In reply to Josh Cogliati from comment #0)
> > The problem seems to be that curl is using a http proxy tunnel for some
> > reason.
> 
> The reason is that you asked curl to use TLS, so curl will not send possibly
> sensitive data to the proxy in plain text:
> 
> https://github.com/bagder/curl/blob/bb0acba6/lib/url.c#L5584
> 
> You need a proxy server that supports CONNECT to let curl speak HTTPS over
> it.

Based on my understanding of the output, CONNECT works with curl+openssl, and wget, but not with curl+nss.  Do you have any hints as to how I can better debug this problem?

PR_END_OF_FILE_ERROR usually means that peer (either the remote server or proxy) closed the connection.  It must be something specific to your proxy or network.  Is there any firewall in place?

Your example works fine on my Fedora 22 system if I use a local squid instance with default configuration.  If you are able to repeat the problem against a publicly available proxy or a locally installable proxy, I will give it a try.

(In reply to Kamil Dudka from comment #5)
> PR_END_OF_FILE_ERROR usually means that peer (either the remote server or
> proxy) closed the connection.  It must be something specific to your proxy
> or network.  Is there any firewall in place?
> 
> Your example works fine on my Fedora 22 system if I use a local squid
> instance with default configuration.  If you are able to repeat the problem
> against a publicly available proxy or a locally installable proxy, I will
> give it a try.

Hm, I also get the same effect when I set up squid locally with the proxy as cache_peer, but that doesn't help you debug it.  It is possibly the proxy is wrong, but I don't have a really good understanding of what is different about curl+nss versus things that work like curl+openssl or wget or firefox.

Could you please capture the communication with your proxy by tcpdump and attach it to this bug?

# tcpdump -i any -w bz1248829.pcap -s 0 port 8080

Josh sent me privately network captures of both the working and non-working scenarios.  After comparing them with each other, we figured out that this is  just a matter of enabled cipher-suites.  Josh confirmed that curl successfully connects over the proxy with --ciphers ecdhe_rsa_aes_128_gcm_sha_256.

$ rpm -aq nss\* | sort -V
nss-3.19.2-1.0.fc21.x86_64
nss-devel-3.19.2-1.0.fc21.x86_64
nss-mdns-0.10-15.fc21.x86_64
nss-softokn-3.19.2-1.0.fc21.x86_64
nss-softokn-devel-3.19.2-1.0.fc21.x86_64
nss-softokn-freebl-3.19.2-1.0.fc21.x86_64
nss-softokn-freebl-devel-3.19.2-1.0.fc21.x86_64
nss-sysinit-3.19.2-1.0.fc21.x86_64
nss-tools-3.19.2-1.0.fc21.x86_64
nss-util-3.19.2-1.0.fc21.x86_64
nss-util-devel-3.19.2-1.0.fc21.x86_64

*** This bug has been marked as a duplicate of bug 1185708 ***