Hello,
 There is a difference in the gnutls certificate validation in F20 and F21. In F20 the file /etc/pki/tls/cert.pem is used to load the trusted CA certificates, while in F21 the p11-kit trust module is queried (shown with p11tool --list-tokens). That shouldn't have had much differences though, as the trust module is the one that generates the /etc/pki/tls/cert.pem in the first place.

However, I'm confused by the test case. In F20 connecting to www.amazon.com succeeds (or at least it should). Have you tweaked the trusted certificate database? (did you run update-ca-trust after then?)

(In reply to Nikos Mavrogiannopoulos from comment #1)
> However, I'm confused by the test case. In F20 connecting to www.amazon.com
> succeeds (or at least it should).

It succeeds unless I install ca-certificates-2014.2.1-1.0.fc20 from updates-testing (to match the version used in F21/F22), in which case it fails. Installing or downgrading this package is sufficient to cause gnutls-cli in F20 to reject or accept the certificate from www.amazon.com. In F21/F22 it makes no difference to gnutls-cli, but does to glib.

> Have you tweaked the trusted certificate
> database? (did you run update-ca-trust after then?)

This is run in %post when upgrading or downgrading the ca-certificates package.

I suspect the issue is that glib uses gnutls_certificate_set_x509_trust_file() and sets /etc/pki/tls/cert.pem, while gnutls-cli in F21 uses gnutls_certificate_set_x509_system_trust() which uses the p11-kit trust module.

However, even in that case ca-certificates should have provided the same set of anchors. That's why I'm assigning this issue to ca-certificates.

It seems it's easy to reproduce:

$ sudo dnf install https://kojipkgs.fedoraproject.org//packages/ca-certificates/2014.2.1/1.0.fc20/noarch/ca-certificates-2014.2.1-1.0.fc20.noarch.rpm

#using gnutls 3.3.x or the shipped in f20
$ gnutls-cli www.amazon.com --x509cafile /etc/pki/tls/cert.pem 
- Status: The certificate is NOT trusted. The certificate issuer is unknown. 
*** Verifying server certificate failed...
*** Fatal error: Error in the certificate.
*** Handshake has failed
GnuTLS error: Error in the certificate.

#using gnutls 3.3.x
$ ./gnutls-cli www.amazon.com --x509cafile pkcs11:
...
- Handshake was completed
- Simple Client Mode:


So indeed, there is an issue in ca-certificates-2014.2.1-1.0.fc20.noarch.rpm

The issue is that "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary Certification Authority" is missing from the trusted for TLS authorities.

However, it is present for the trusted for email authorities. That's why the version of gnutls which uses p11-kit works.

There are two issues:
1. the p11-kit trust module includes all authorities and relies on the user checking the key purpose extension for verification. Since the certificate in question doesn't set the key purpose it is considered trusted for TLS. Hence, the discrepancy between using the tls/cert.pem file and p11-kit trust module.

2. Amazon.com sends this certificate as its last: "Subject: C=US,O=VeriSign\, Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5"

That is signed by "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary Certification Authority", which was removed.

GnuTLS in that case will check whether "Subject: C=US,O=VeriSign\, Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5" is in the trusted list and matches the certificate sent by amazon.

However our bundled, doesn't match the one sent by amazon. Amazon's sent G5 SHA1 fingerprint is: f4a80a0cd1e6cf190b8cbc6fbc991711d482c9d0, while our bundled G5 has SHA1: 4eb6d578499b1ccf5f581ead56be3d9b6744a5e5

For some reason we ship a newer version of G5 which is different than the one amazon sends. A fix could be to send the old version of G5 as well (the non-self-signed one).

(In reply to Nikos Mavrogiannopoulos from comment #5)
> For some reason we ship a newer version of G5 which is different than the
> one amazon sends. A fix could be to send the old version of G5 as well (the
> non-self-signed one).

s/to send the old/to bundle the old/

Thanks Nikos.  So the root certificate is valid for TLS, but is missing from tls/cert.pem? Am I correct in saying that applications are allowed to assume that all valid TLS certificates are available in tls/cert.pem? Is this a bug in ca-certificates or p11-kit?

I wonder why we have a different version of the G5 cert.

(In reply to Michael Catanzaro from comment #7)
> Thanks Nikos.  So the root certificate is valid for TLS, but is missing from
> tls/cert.pem? 

Yes. It is one of the certificates that were removed because they were 1024 bits. But it was removed only from the trusted for TLS list, not the trusted for email list.

> Am I correct in saying that applications are allowed to assume
> that all valid TLS certificates are available in tls/cert.pem? Is this a bug
> in ca-certificates or p11-kit?

I'm not sure (adding Stef).

(In reply to Nikos Mavrogiannopoulos from comment #8)
> Yes. It is one of the certificates that were removed because they were 1024
> bits. But it was removed only from the trusted for TLS list, not the trusted
> for email list.

Then it seems like Firefox should also not accept this cert for TLS?

(As a practical matter, my main concern is that Firefox can display amazon.com and Epiphany cannot.)

(In reply to Michael Catanzaro from comment #9)
> (In reply to Nikos Mavrogiannopoulos from comment #8)
> > Yes. It is one of the certificates that were removed because they were 1024
> > bits. But it was removed only from the trusted for TLS list, not the trusted
> > for email list.
> 
> Then it seems like Firefox should also not accept this cert for TLS?

Why? Firefox uses NSS which does use a totally different verification strategy (and database). I just described what gnutls uses.

(In reply to Nikos Mavrogiannopoulos from comment #8)
> (In reply to Michael Catanzaro from comment #7)
> > Am I correct in saying that applications are allowed to assume
> > that all valid TLS certificates are available in tls/cert.pem? Is this a bug
> > in ca-certificates or p11-kit?
> 
> I'm not sure (adding Stef).

Not sure what "valid TLS certificates" means, exactly.

But tls/cert.pem is a symlink to /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

That file contains all root CA anchors trusted to verify TLS servers.

There is also intermediate certificate trust information that's not included in the extracted files. OpenSSL and GnuTLS have (at the current time at least) no way to consume intermediate certificate trust information from files.

(In reply to Stef Walter from comment #11)
> (In reply to Nikos Mavrogiannopoulos from comment #8)
> > (In reply to Michael Catanzaro from comment #7)
> > > Am I correct in saying that applications are allowed to assume
> > > that all valid TLS certificates are available in tls/cert.pem? Is this a bug
> > > in ca-certificates or p11-kit?
> > 
> > I'm not sure (adding Stef).
> 
> Not sure what "valid TLS certificates" means, exactly.
> 
> But tls/cert.pem is a symlink to
> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
> 
> That file contains all root CA anchors trusted to verify TLS servers.

The point was that using p11-kit trust module both tls-ca-bundle is given to the application as well as the email-ca-bundle. The application at that point has no way to distinguish them. However maybe we should split the issues, as that is not directly related to the ca-certificates issue.

(In reply to Nikos Mavrogiannopoulos from comment #12)
> The point was that using p11-kit trust module both tls-ca-bundle is given to
> the application as well as the email-ca-bundle. The application at that
> point has no way to distinguish them. However maybe we should split the
> issues, as that is not directly related to the ca-certificates issue.

Older applications that don't support looking stuff up in the trust store directlty, should only use *one* of the bundle files, not both.

The one that it uses depends on what the application is doing. If the application is connecting to a TLS server, use tls-ca-bundle.pem. If the application is doing S/MIME, then it should use email-ca-bundle.pem.

Let's separate the issues, to clean up the mess in this bug report. I'll fill the issue on p11-kit separately.

The issue in this report is that "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary Certification Authority" is missing from the trusted for TLS authorities.

Amazon.com sends this certificate as its last: "Subject: C=US,O=VeriSign\, Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5"

That is signed by "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary Certification Authority", which was removed.

GnuTLS in that case will check whether "Subject: C=US,O=VeriSign\, Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5" is in the trusted list and matches the certificate sent by amazon.

However our bundled, doesn't match the one sent by amazon. Amazon's sent G5 SHA1 fingerprint is: f4a80a0cd1e6cf190b8cbc6fbc991711d482c9d0, while our bundled G5 has SHA1: 4eb6d578499b1ccf5f581ead56be3d9b6744a5e5

For some reason we ship a newer version of G5 which is different than the one amazon sends. A fix could be to send the old version of G5 as well (the non-self-signed one).

(In reply to Nikos Mavrogiannopoulos from comment #5)
> 2. Amazon.com sends this certificate as its last: "Subject:
> C=US,O=VeriSign\, Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc.
> - For authorized use only,CN=VeriSign Class 3 Public Primary Certification
> Authority - G5"
> 
> That is signed by "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary
> Certification Authority", which was removed.
> 
> GnuTLS in that case will check whether "Subject: C=US,O=VeriSign\,
> Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized
> use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5" is
> in the trusted list and matches the certificate sent by amazon.
> 
> However our bundled, doesn't match the one sent by amazon. Amazon's sent G5
> SHA1 fingerprint is: f4a80a0cd1e6cf190b8cbc6fbc991711d482c9d0, while our
> bundled G5 has SHA1: 4eb6d578499b1ccf5f581ead56be3d9b6744a5e5
> 
> For some reason we ship a newer version of G5 which is different than the
> one amazon sends. A fix could be to send the old version of G5 as well (the
> non-self-signed one).

It's easy to get confused here:

The certificate with subject G5 that is sent by the amazon server is an intermediate. The intermediate has a different issuer, pointing to the removed root.

Our bundled cert with same subject G5 is a self signed root CA, with identical subject and issuer.

They are two different certs, that's why they of course have different SHA1.

(In reply to Nikos Mavrogiannopoulos from comment #8)
> (In reply to Michael Catanzaro from comment #7)
> > Thanks Nikos.  So the root certificate is valid for TLS, but is missing from
> > tls/cert.pem? 
> 
> Yes. It is one of the certificates that were removed because they were 1024
> bits. But it was removed only from the trusted for TLS list, not the trusted
> for email list.

To clarify:

The old root is still shipped in our "master" (source) list.

Each shipped root may come with three independent trust flags. The old root had its trust flag for SSL/TLS removed. It's still trusted for email.

The intention is that update-ca-trust/p11-kit-trust reads the master/source list, and when it creates the "extracted" bundle files, it should omit the root from the tls bundle, but include it in the email bundle.

In my opinion, applications that use p11-kit-trust MUST look at the trust flags when making a decision about "trusted for SSL/TLS".

(In reply to Nikos Mavrogiannopoulos from comment #14)
> Let's separate the issues, to clean up the mess in this bug report. I'll
> fill the issue on p11-kit separately.
> 
> The issue in this report is that "C=US,O=VeriSign\, Inc.,OU=Class 3 Public
> Primary Certification Authority" is missing from the trusted for TLS
> authorities.
> 
> Amazon.com sends this certificate as its last: "Subject: C=US,O=VeriSign\,
> Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized
> use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5"
> 
> That is signed by "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary
> Certification Authority", which was removed.

right, removed from the "list of certs trusted for SSL/TLS" (but still included as trusted for issued "email" certs.


> GnuTLS in that case will check whether "Subject: C=US,O=VeriSign\,
> Inc.,OU=VeriSign Trust Network,OU=(c) 2006 VeriSign\, Inc. - For authorized
> use only,CN=VeriSign Class 3 Public Primary Certification Authority - G5" is
> in the trusted list and matches the certificate sent by amazon.

There is a certificate with this subject in our trusted list.

Are you searching for a trust store certificate that is IDENTICAL to the certificate sent by the server? I think that would be wrong.

I think what you must do is:

Search the trust store for ANY certificate that matches exactly the given subject. If you find a certificate that is self-signed (same issuer) and marked as trusted, you should succeed.


> However our bundled, doesn't match the one sent by amazon. Amazon's sent G5
> SHA1 fingerprint is: f4a80a0cd1e6cf190b8cbc6fbc991711d482c9d0, while our
> bundled G5 has SHA1: 4eb6d578499b1ccf5f581ead56be3d9b6744a5e5

I think that should be irrelevant, as I explained in the previous comment.


> For some reason we ship a newer version of G5 which is different than the
> one amazon sends. A fix could be to send the old version of G5 as well (the
> non-self-signed one).

It's a common practice that a CA will issue multiple certificates with the same subject, but different issuers. A common scenario is "cross-signing", or for migration purposes (when introducing a new root, allow clients to temporarily find a chain to another pre-existing root).

(In reply to Kai Engert (:kaie) from comment #16)
> In my opinion, applications that use p11-kit-trust MUST look at the trust
> flags when making a decision about "trusted for SSL/TLS".

They were not made available, so it is no possible to check them. As I understand there is on-going work, but as it is now in F20 the flags are not available.

(In reply to Kai Engert (:kaie) from comment #17)

> I think what you must do is:
> Search the trust store for ANY certificate that matches exactly the given
> subject. If you find a certificate that is self-signed (same issuer) and
> marked as trusted, you should succeed.

There is no cryptographic binding of the key in the DN. At minimum the match of the key between the suggested by the server and the bundled CA should be performed, or in the end the provided by the user chain becomes irrelevant (and one may end up verifying ECDSA signature using an RSA key). When combined with key matching it looks reasonable and is already added in gnutls 3.3.x, but I'm not sure whether it should be backported to 3.1.x.
 
> > However our bundled, doesn't match the one sent by amazon. Amazon's sent G5
> > SHA1 fingerprint is: f4a80a0cd1e6cf190b8cbc6fbc991711d482c9d0, while our
> > bundled G5 has SHA1: 4eb6d578499b1ccf5f581ead56be3d9b6744a5e5
> I think that should be irrelevant, as I explained in the previous comment.

It could be, but by being irrelevant we accept that we will not be able to connect to amazon.com using "Basic Path Validation" as in http://tools.ietf.org/html/rfc5280#section-6

(In reply to Kai Engert (:kaie) from comment #17)

> > That is signed by "C=US,O=VeriSign\, Inc.,OU=Class 3 Public Primary
> > Certification Authority", which was removed.
> right, removed from the "list of certs trusted for SSL/TLS" (but still
> included as trusted for issued "email" certs.

A related issue, is why is that? Why does a 1024-bit CA certificate is considered trusted to certify e-mails but not web sites? We haven't set any requirements that set a higher bar for the web than for e-mail.

Fixed in F21.

(to clarify: the parts that were related to gnutls are fixed in F21)