Bug 1248829 - dnf can't get metalink thru proxy
dnf can't get metalink thru proxy
Status: CLOSED DUPLICATE of bug 1185708
Product: Fedora
Classification: Fedora
Component: nss (Show other bugs)
21
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Elio Maldonado Batiz
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-07-30 18:14 EDT by Josh Cogliati
Modified: 2017-04-21 06:19 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-19 11:00:59 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Josh Cogliati 2015-07-30 18:14:59 EDT
Description of problem:
dnf cannot retrieve the metalink file through a proxy.

# dnf makecache fast
Error: Failed to synchronize cache for repo 'fedora' from 'https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64': Cannot prepare internal mirrorlist: Curl error (35): SSL connect error for https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64 [Encountered end of file]


Version-Release number of selected component (if applicable):
# rpm -q dnf
dnf-1.0.0-1.fc22.noarch


How reproducible:
Very.

Steps to Reproduce:
1. dnf makecache fast
2.
3.

Actual results:
Error: Failed to synchronize cache for repo 'fedora' from 'https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64': Cannot prepare internal mirrorlist: Curl error (35): SSL connect error for https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64 [Encountered end of file]


Expected results:
It would work.


Additional info:
The problem seems to be that curl is using a http proxy tunnel for some reason.
(as if --proxytunnel was used)
In order to connect to the internet, I have a proxy.
If I do (proxyip replaced with <proxyip>):
curl -v 'https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64'

*   Trying <proxyip>...
* Connected to <proxyip> (<proxyip>) port 8080 (#0)
* Establish HTTP proxy tunnel to mirrors.fedoraproject.org:443
* Proxy auth using Basic with user 'cogljj'
> CONNECT mirrors.fedoraproject.org:443 HTTP/1.1
> Host: mirrors.fedoraproject.org:443
> Proxy-Authorization: Basic <data>
> User-Agent: curl/7.40.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* NSS error -5938 (PR_END_OF_FILE_ERROR)
* Encountered end of file
* Closing connection 0
curl: (35) Encountered end of file


wget works fine:

# wget -S 'https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64'
--2015-07-30 16:12:04--  https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64
Connecting to <proxyip>:8080... connected.
Proxy request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Thu, 30 Jul 2015 22:12:04 GMT
  Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
  Content-Length: 16226
  AppTime: D=735111
  AppServer: proxy04.fedoraproject.org
  Content-Type: application/metalink+xml
  Keep-Alive: timeout=15, max=500
  Connection: Keep-Alive
Length: 16226 (16K) [application/metalink+xml]
Saving to: ‘metalink?repo=fedora-22&arch=x86_64.1’

metalink?repo=fedor 100%[=====================>]  15.85K  6.59KB/s   in 2.4s   

2015-07-30 16:12:07 (6.59 KB/s) - ‘metalink?repo=fedora-22&arch=x86_64.1’ saved [16226/16226]
Comment 1 Josh Cogliati 2015-07-31 16:05:41 EDT
Hm.  It appears to be a problem with the nss library.

(Oh, and this happens in Fedora 21, and until I solve it for Fedora 22, I can't do much in Fedora 22.)

$ rpm -q nss-devel openssl-devel
nss-devel-3.19.2-1.0.fc21.x86_64
openssl-devel-1.0.1k-11.fc21.x86_64

If I build curl with nss, I get the same behavior, but if I build it with openssl, it works.

./configure --prefix=$HOME/local/curl_install --with-openssl

$ curl --version
curl 7.43.0 (x86_64-unknown-linux-gnu) libcurl/7.43.0 OpenSSL/1.0.1k zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets 
$ curl -v 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-f21&arch=x86_64'
*   Trying <proxyip>...
* Connected to <proxyip> (<proxyip>) port 8080 (#0)
* Establish HTTP proxy tunnel to mirrors.fedoraproject.org:443
* Proxy auth using Basic with user 'cogljj'
> CONNECT mirrors.fedoraproject.org:443 HTTP/1.1
> Host: mirrors.fedoraproject.org:443
> Proxy-Authorization: Basic <data>
> User-Agent: curl/7.43.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* NPN, no overlap, use HTTP1.1
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Unknown (67):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
* 	 subject: C=US; ST=North Carolina; L=Raleigh; O=Red Hat Inc.; CN=*.fedoraproject.org
* 	 start date: 2014-04-22 00:00:00 GMT
* 	 expire date: 2017-04-26 12:00:00 GMT
* 	 subjectAltName: mirrors.fedoraproject.org matched
* 	 issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
* 	 SSL certificate verify ok.
> GET /metalink?repo=updates-released-f21&arch=x86_64 HTTP/1.1
> Host: mirrors.fedoraproject.org
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Fri, 31 Jul 2015 19:59:47 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
< Content-Length: 10039
< AppTime: D=261752
< AppServer: proxy03.fedoraproject.org
< Content-Type: application/metalink+xml
< 
<?xml version="1.0" encoding="utf-8"?>
Comment 2 Michal Luscon 2015-08-11 08:17:43 EDT
Ok, this looks like a curl issue -> reassigning.
Comment 3 Kamil Dudka 2015-08-11 10:50:47 EDT
(In reply to Josh Cogliati from comment #0)
> The problem seems to be that curl is using a http proxy tunnel for some
> reason.

The reason is that you asked curl to use TLS, so curl will not send possibly sensitive data to the proxy in plain text:

https://github.com/bagder/curl/blob/bb0acba6/lib/url.c#L5584

You need a proxy server that supports CONNECT to let curl speak HTTPS over it.
Comment 4 Josh Cogliati 2015-08-11 14:12:56 EDT
(In reply to Kamil Dudka from comment #3)
> (In reply to Josh Cogliati from comment #0)
> > The problem seems to be that curl is using a http proxy tunnel for some
> > reason.
> 
> The reason is that you asked curl to use TLS, so curl will not send possibly
> sensitive data to the proxy in plain text:
> 
> https://github.com/bagder/curl/blob/bb0acba6/lib/url.c#L5584
> 
> You need a proxy server that supports CONNECT to let curl speak HTTPS over
> it.

Based on my understanding of the output, CONNECT works with curl+openssl, and wget, but not with curl+nss.  Do you have any hints as to how I can better debug this problem?
Comment 5 Kamil Dudka 2015-08-12 04:22:43 EDT
PR_END_OF_FILE_ERROR usually means that peer (either the remote server or proxy) closed the connection.  It must be something specific to your proxy or network.  Is there any firewall in place?

Your example works fine on my Fedora 22 system if I use a local squid instance with default configuration.  If you are able to repeat the problem against a publicly available proxy or a locally installable proxy, I will give it a try.
Comment 6 Josh Cogliati 2015-08-14 12:22:09 EDT
(In reply to Kamil Dudka from comment #5)
> PR_END_OF_FILE_ERROR usually means that peer (either the remote server or
> proxy) closed the connection.  It must be something specific to your proxy
> or network.  Is there any firewall in place?
> 
> Your example works fine on my Fedora 22 system if I use a local squid
> instance with default configuration.  If you are able to repeat the problem
> against a publicly available proxy or a locally installable proxy, I will
> give it a try.

Hm, I also get the same effect when I set up squid locally with the proxy as cache_peer, but that doesn't help you debug it.  It is possibly the proxy is wrong, but I don't have a really good understanding of what is different about curl+nss versus things that work like curl+openssl or wget or firefox.
Comment 7 Kamil Dudka 2015-08-17 09:07:53 EDT
Could you please capture the communication with your proxy by tcpdump and attach it to this bug?

# tcpdump -i any -w bz1248829.pcap -s 0 port 8080
Comment 8 Kamil Dudka 2015-08-19 11:00:59 EDT
Josh sent me privately network captures of both the working and non-working scenarios.  After comparing them with each other, we figured out that this is  just a matter of enabled cipher-suites.  Josh confirmed that curl successfully connects over the proxy with --ciphers ecdhe_rsa_aes_128_gcm_sha_256.

$ rpm -aq nss\* | sort -V
nss-3.19.2-1.0.fc21.x86_64
nss-devel-3.19.2-1.0.fc21.x86_64
nss-mdns-0.10-15.fc21.x86_64
nss-softokn-3.19.2-1.0.fc21.x86_64
nss-softokn-devel-3.19.2-1.0.fc21.x86_64
nss-softokn-freebl-3.19.2-1.0.fc21.x86_64
nss-softokn-freebl-devel-3.19.2-1.0.fc21.x86_64
nss-sysinit-3.19.2-1.0.fc21.x86_64
nss-tools-3.19.2-1.0.fc21.x86_64
nss-util-3.19.2-1.0.fc21.x86_64
nss-util-devel-3.19.2-1.0.fc21.x86_64

*** This bug has been marked as a duplicate of bug 1185708 ***

Note You need to log in before you can comment on or make changes to this bug.