Hide Forgot
Description of problem: Curl does not honour -capath any more. Version-Release number of selected component (if applicable): curl-7.19.7-16.el6.x86_64 How reproducible: 100% Steps to Reproduce: #! /bin/sh set -e cadir=$(mktemp -d) cd $cadir # Fetch the root certificate SPI, Inc. Certificate Authority # This is a CA which is not recognised by RHEL by default wget http://www.spi-inc.org/ca/spi-cacert.crt # Make sure OpenSSL can find it ln -s spi-cacert.crt 56e29e75.0 ln -s spi-cacert.crt 56e29e75.1 # Now access a site whose certificate is signed by the SPI CA. strace -f -s128 -o trace.txt -- \ curl --verbose --capath $cadir https://trac.oftc.net # Any attempt to use the capath? grep $cadir trace.txt Actual results: --2011-01-14 14:27:35-- http://www.spi-inc.org/ca/spi-cacert.crt Resolving www.spi-inc.org... 89.16.189.61 Connecting to www.spi-inc.org|89.16.189.61|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2854 (2.8K) [application/x-x509-ca-cert] Saving to: “spi-cacert.crt” 0K .. 100% 74.8M=0s 2011-01-14 14:27:35 (74.8 MB/s) - “spi-cacert.crt” saved [2854/2854] * About to connect() to trac.oftc.net port 443 (#0) * Trying 72.32.146.138... connected * Connected to trac.oftc.net (72.32.146.138) port 443 (#0) * Initializing NSS with certpath: /etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: /tmp/tmp.DhB1IwnNhv * Certificate is signed by an untrusted issuer: 'E=support,CN=servers.ca.oftc.net,OU=certification authority for servers,O=Open and Free Technology Community' * NSS error -8172 * Closing connection #0 * Peer certificate cannot be authenticated with known CA certificates curl: (60) Peer certificate cannot be authenticated with known CA certificates More details here: http://curl.haxx.se/docs/sslcerts.html curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option. If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL). If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. Expected results: -capath being honoured, like it is on RHEL5: --2011-01-14 14:29:12-- http://www.spi-inc.org/ca/spi-cacert.crt Resolving www.spi-inc.org... 89.16.189.61 Connecting to www.spi-inc.org|89.16.189.61|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2854 (2.8K) [application/x-x509-ca-cert] Saving to: `spi-cacert.crt' 0K .. 100% 36.8M=0s 2011-01-14 14:29:12 (36.8 MB/s) - `spi-cacert.crt' saved [2854/2854] * About to connect() to trac.oftc.net port 443 * Trying 72.32.146.138... connected * Connected to trac.oftc.net (72.32.146.138) port 443 * successfully set certificate verify locations: * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: /tmp/tmp.JzhCaj4366 * SSLv2, Client hello (1): SSLv3, TLS handshake, Server hello (2): SSLv3, TLS handshake, CERT (11): SSLv3, TLS handshake, Server key exchange (12): SSLv3, TLS handshake, Server finished (14): SSLv3, TLS handshake, Client key exchange (16): SSLv3, TLS change cipher, Client hello (1): SSLv3, TLS handshake, Finished (20): SSLv3, TLS change cipher, Client hello (1): SSLv3, TLS handshake, Finished (20): SSL connection using DHE-RSA-AES256-SHA * Server certificate: * subject: /O=Open and Free Technology Community/CN=trac.oftc.net/emailAddress=support * start date: 2010-05-04 14:58:35 GMT * expire date: 2011-05-04 14:58:35 GMT * common name: trac.oftc.net (matched) * issuer: /O=Open and Free Technology Community/OU=certification authority for servers/CN=servers.ca.oftc.net/emailAddress=support * SSL certificate verify ok. > GET / HTTP/1.1 > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: trac.oftc.net > Accept: */* > < HTTP/1.1 302 Found < Date: Fri, 14 Jan 2011 13:29:14 GMT < Server: Apache < Location: http://www.oftc.net/oftc/CodingProjects < Content-Length: 287 < Content-Type: text/html; charset=iso-8859-1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 287 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 100 287 100 287 0 0 174 0 0:00:01 0:00:01 --:--:-- 41000* Connection #0 to host trac.oftc.net left intact * Closing connection #0 * SSLv3, TLS alert, Client hello (1): <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://www.oftc.net/oftc/CodingProjects">here</a>.</p> <hr> <address>Apache Server at trac.oftc.net Port 443</address> </body></html> 4371 execve("/usr/bin/curl", ["curl", "--verbose", "--capath", "/tmp/tmp.JzhCaj4366", "https://trac.oftc.net"], [/* 22 vars */]) = 0 4371 write(2, " CAfile: /etc/pki/tls/certs/ca-bundle.crt\n CApath: /tmp/tmp.JzhCaj4366\n", 73) = 73 4371 stat("/tmp/tmp.JzhCaj4366/56e29e75.0", {st_mode=S_IFREG|0664, st_size=2854, ...}) = 0 4371 open("/tmp/tmp.JzhCaj4366/56e29e75.0", O_RDONLY) = 4 4371 stat("/tmp/tmp.JzhCaj4366/56e29e75.1", {st_mode=S_IFREG|0664, st_size=2854, ...}) = 0 4371 open("/tmp/tmp.JzhCaj4366/56e29e75.1", O_RDONLY) = 4 Additional info: Reproducible with curl-7.21.3-2.fc15.x86_64 in Fedora Rawhide as well.
Does it connect successfully if you pass the CA certificate directly with --cacert ?
Yes, it does.
Then we need to improve the code that enumerates files in the given directory, and also update the man page -- it talks about OpenSSL hashing utility, which makes no sense for NSS powered curl.
Created attachment 473876 [details] proposed fix Rob, could you please review the attached patch?
Comment on attachment 473876 [details] proposed fix I approve of the re-working of the code, the result is a lot more readable. I don't see any significant changes to the behavior though, how does this change make it work that the previous code didn't?
The fact that the flaw had not been visible (without gdb) actually forced me to make the code more readable. The main problem was the if/else sequence here: if(!data->set.ssl.verifypeer) /* skip the verifying of the peer */ ; else if(data->set.ssl.CAfile) { ... } else if(data->set.ssl.CApath) { ... } The legacy code treated the CAfile and CApath options mutually exclusive, which is wrong. CAfile fallbacks to the default CA bundle and therefore CApath had been never used.
Ray, does the patch solve the problem for you?
Comment on attachment 473876 [details] proposed fix Ok, makes sense now.
(In reply to comment #8) > Ray, does the patch solve the problem for you? Yes, it does. Nice work!
pushed upstream: https://github.com/bagder/curl/commit/fc77790
It seems that the fix have introduced bug 586355 again. The only diference is that curl doesn't timeout. I have tested bug 586355 with curl-7.19.7-18.el6.x86_64: :: [ PASS ] :: Running curl with --connect-timeout 7 * About to connect() to localhost port 4433 (#0) * Trying ::1... connected * Connected to localhost (::1) port 4433 (#0) * Initializing NSS with certpath: /etc/pki/nssdb * CAfile: ./myca.crt CApath: none * NSS error -5990 * Closing connection #0 * SSL connect error curl: (35) SSL connect error :: [ PASS ] :: File 'curl.out' should contain 'SSL connect error' :: [ PASS ] :: Save time after running curl :: [06:05:36] :: curl was waiting 7 seconds ..and with curl-7.19.7-19.el6.x86_64 that adds: - fix handling of CURLOPT_CAPATH in libcurl (#669702) :: [ FAIL ] :: Running curl with --connect-timeout 7 (Expected 28,35, got 56) * About to connect() to localhost port 4433 (#0) * Trying ::1... connected * Connected to localhost (::1) port 4433 (#0) * Initializing NSS with certpath: /etc/pki/nssdb * CAfile: ./myca.crt CApath: none * NSS error -5990 > GET /index.html HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > Host: localhost:4433 > Accept: */* > % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:13 --:--:-- 0* Closing connection #0 * Failure when receiving data from the peer curl: (56) Failure when receiving data from the peer :: [ FAIL ] :: File 'curl.out' should contain 'SSL connect error' :: [ PASS ] :: Save time after running curl :: [06:07:42] :: curl was waiting 181 seconds After 180 seconds the peer (openssl server) has been terminated, therefore the failure. Shortened test scenario is at https://bugzilla.redhat.com/show_bug.cgi?id=586355#c6
Ups, I tried ^ ^ more time, in case of 180 second limit also the Time Spent column contains 0:03:00.
Perhaps the amount of time should be updated? The loading of CA certs may take some time (and memory) since the PEM reader module is not much optimized yet...
Which amount of time do you mean exactly? Karel says that the new curl was waiting for 180s and it didn't even then timeout.
(In reply to comment #16) > :: [ FAIL ] :: Running curl with --connect-timeout 7 (Expected 28,35, got > 56) > * About to connect() to localhost port 4433 (#0) > * Trying ::1... connected > * Connected to localhost (::1) port 4433 (#0) > * Initializing NSS with certpath: /etc/pki/nssdb > * CAfile: ./myca.crt > CApath: none > * NSS error -5990 So far so good, -5990 means PR_IO_TIMEOUT_ERROR. > > GET /index.html HTTP/1.1 > > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > > Host: localhost:4433 > > Accept: */* This ^^^ looks suspicious. curl is talking to an already dead connection.
The patch is indeed broken. Thanks for catching it. The 'error' variable is mistakenly nullified at this point: https://github.com/bagder/curl/commit/fc77790#L3R1302 We need to improve the patch such that 'error' stays set to CURLE_SSL_CONNECT_ERROR.
(In reply to comment #21) s/error/curlerr/
pushed a fix upstream: https://github.com/bagder/curl/commit/7aa2d10
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: libcurl ignored the CA path provided in CURLOPT_CAPATH and consequently curl ignored the "--capath" argument provided. This is fixed so that libcurl now uses the value provided with the the "--capath" argument.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2011-0573.html