Bug 1861495 - F33/Rawhide: crypto-policies update causes JSS's OCSP Leaf and Chain checking to fail with Unknown Error
Summary: F33/Rawhide: crypto-policies update causes JSS's OCSP Leaf and Chain checking...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: nss
Version: 33
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: ---
Assignee: Bob Relyea
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1858017
TreeView+ depends on / blocked
 
Reported: 2020-07-28 18:30 UTC by Alex Scheel
Modified: 2021-11-09 16:00 UTC (History)
9 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2021-11-09 16:00:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
OCSP response file on badssl.com (471 bytes, application/octet-stream)
2020-08-04 16:59 UTC, Tomas Mraz
no flags Details
Skip the signature check if the cert is self-signed. (3.05 KB, patch)
2020-10-20 23:07 UTC, Bob Relyea
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Mozilla Foundation 1672291 0 P1 RESOLVED libpkix OCSP failures on SHA1 self-signed root certs when SHA1 signatures are disabled. 2021-01-14 17:03:00 UTC

Description Alex Scheel 2020-07-28 18:30:40 UTC
Description of problem:

A recent update to crypto-policies (probably related to the Change proposal) has caused JSS's BadSSL regression test (with LEAF_AND_CHAIN OCSP validation) to fail with an unexpected error message.

https://pipelines.actions.githubusercontent.com/0v9771Hs5cPQUVTnXwXZ0gqk9opLP1Q0hyLl3htdE40JtWxNLO/_apis/pipelines/1/runs/1112/signedlogcontent/7?urlExpires=2020-07-28T17%3A59%3A21.2640914Z&urlSigningMethod=HMACV1&urlSignature=GO6A3x6ng9P67NfW7beuipxB%2BT74SB5C7T9smLMtgjk%3D

Downgrading to the version of packages in F32 seems to work fine.


I'm inclined this is something wrong on my end, but I find it weird that the behavior only manifests on OCSP checks and only just recently started to fail. 

However, this could also be an issue with the remote OCSP server.

https://github.com/dogtagpki/jss/blob/master/org/mozilla/jss/tests/BadSSL.java#L94-L96

Version-Release number of selected component (if applicable):

crypto-policies             noarch  20200702-1.gitc40cede.fc33  rawhide   56 k
crypto-policies-scripts     noarch  20200702-1.gitc40cede.fc33  rawhide   62 k

How reproducible:

CI consistently fails.

Steps to Reproduce:
1. Pull JSS upstream
2. $ ./tools/run_container.sh fedora_rawhide
3. (Or execute steps in tools/Dockerfile/fedora_rawhide)

Actual results:

BadSSL's LEAF_AND_CHAIN check fails

Expected results:

BadSSL's LEAF_AND_CHAIN should pass like it does on F32 and older.


Additional info:

This might be an issue with how NSS interacts with the new crypto policy.

Comment 1 Alex Scheel 2020-07-28 18:37:20 UTC
Other random comments:

 - OpenSSL will validate OCSP response fine on Fedora Rawhide and F32 (see script below) -- note that OCSP cert needs to be side-loaded.
 - LEAF_AND_CHAIN uses CERT_PKIXVerifyCert -- https://github.com/dogtagpki/jss/blob/master/org/mozilla/jss/ssl/common.c#L1064


-----

#!/bin/bash

hostname="$1"

rm -f certificate.pem chain.pem

openssl s_client -connect "$hostname:443" < /dev/null 2>&1 |  sed -n '/-----BEGIN/,/-----END/p' > certificate.pem
openssl s_client -showcerts -connect "$hostname:443" < /dev/null 2>&1 |  sed -n '/-----BEGIN/,/-----END/p' > chain.pem
URL="$(openssl x509 -noout -ocsp_uri -in certificate.pem)"
echo "OCSP URL: $URL"
HOST="$(awk -F/ '{print $3}' <<< "$URL")"
openssl ocsp -no_nonce -issuer chain.pem -cert certificate.pem -VAfile ocsp.pem -text -url "$URL"

Comment 2 Tomas Mraz 2020-08-03 16:58:56 UTC
I suppose it is related to the SHA1 disablement. That also means that with the FIPS policy it was always broken as well.

It would be best if NSS allowed SHA1 in OCSP verification or if we could somehow specify in the NSS back-end config that SHA1 is enabled for this purpose.

Daiki, is there a way how to disable SHA1 support in signatures in NSS but still allow it in OCSP certificate identifiers?

Comment 3 Tomas Mraz 2020-08-03 16:59:33 UTC
I suppose if you switch to LEGACY, the test suite passes?

Comment 5 Daiki Ueno 2020-08-04 14:28:58 UTC
(In reply to Tomas Mraz from comment #2)

> Daiki, is there a way how to disable SHA1 support in signatures in NSS but
> still allow it in OCSP certificate identifiers?

I think there is no means to do that in the configuration file, but with the library API (NSS_SetAlgorithmPolicy with setBits/clearBits flags).

Comment 6 Tomas Mraz 2020-08-04 16:59:50 UTC
Created attachment 1710381 [details]
OCSP response file on badssl.com

This is what I've got using openssl ocsp on a badssl.com certificate from digicert.

Comment 7 Bob Relyea 2020-08-04 22:15:23 UTC
OK, I can replicate the issue on RHEL8 with vfyserv:


sudo vi /etc/crypto-policies/back-ends/nss.conf # remove SHA1:
export NSS_ENABLE_PKIX_VERIFY=1
vfyserv -o wrong.host.badssl.com



Result:
Connecting to host wrong.host.badssl.com (addr 104.154.89.105) on port 443
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
(pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed
PROBLEM WITH THE CERT CHAIN:
CERT 2. Builtin Object Token:DigiCert Global Root CA [Certificate Authority]:
  ERROR -8180: Peer's Certificate has been revoked.
Error in function PR_Write: -8180
 - Peer's Certificate has been revoked.

Instead of:
Connecting to host wrong.host.badssl.com (addr 104.154.89.105) on port 443
Error in function PR_Write: -12276
 - Unable to communicate securely with peer: requested domain name does not match the server's certificate.


AS far as I can tell, we aren't even getting into the OCSP code.

Comment 8 Bob Relyea 2020-08-04 23:20:31 UTC
Sigh, It looks like the libpkix revocation checking code is trying to validate the signature on the root cert (which is wrong, root certs are implicitly trusted to be correct). Anyway it can't because the DigiCert selfsigned root is signed using SHA1.

Comment 12 Ben Cotton 2020-08-11 13:50:30 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.

Comment 13 Bob Relyea 2020-10-20 23:07:20 UTC
Created attachment 1723047 [details]
Skip the signature check if the cert is self-signed.

Comment 14 Bob Relyea 2020-10-27 00:05:03 UTC
fixed in rawhide.

Comment 15 Alex Scheel 2020-12-17 20:28:34 UTC
Hey Bob, I'm still seeing this on rawhide; only with leaf and chain OCSP validation, and the error message is much nicer now. :-)

Did I need to do something on my side to enable SHA-1 OCSP validation in libpkix? 

    Caused by: javax.net.ssl.SSLHandshakeException: Error duing SSL.ForceHandshake() :: SEC_ERROR_CERT_SIGNATURE_ALGORITHM_DISABLED (-8016)
	    at org.mozilla.jss.ssl.javax.JSSEngineReferenceImpl.updateHandshakeState(JSSEngineReferenceImpl.java:1037)
	    at org.mozilla.jss.ssl.javax.JSSEngineReferenceImpl.unwrap(JSSEngineReferenceImpl.java:1212)
	    at org.mozilla.jss.ssl.javax.JSSSocketChannel.read(JSSSocketChannel.java:274)
	    ... 10 more


(Note the error occurs during SSL_ForceHandshake because the error propagates up from the custom cert validation handler we install and because we're using NSS's internal cert validation hook rather than an external Java one here.)

This is on rawhide:

-- Checking for module 'nss'
--   Found nss, version 3.59.0


Thanks!

Comment 16 Bob Relyea 2020-12-17 21:16:20 UTC
which nss build are you seeing this?

Comment 17 Bob Relyea 2020-12-17 21:18:34 UTC
Which minor build number (nss-3.59.0-?). There's an issue with nss-3.59.0-1. should be fixed in nss-3.59.0-2

Comment 18 Alex Scheel 2020-12-17 21:24:51 UTC
Ah sorry, its all the latest of everything:

 
(162/221): nspr-4.29.0-9.fc34.x86_64.rpm        2.5 MB/s | 142 kB     00:00    
(163/221): nspr-devel-4.29.0-9.fc34.x86_64.rpm  2.9 MB/s | 175 kB     00:00    
(164/221): nss-devel-3.59.0-2.fc34.x86_64.rpm   2.4 MB/s | 207 kB     00:00    
(165/221): nss-3.59.0-2.fc34.x86_64.rpm         7.1 MB/s | 682 kB     00:00    
(166/221): nss-softokn-3.59.0-2.fc34.x86_64.rpm 7.6 MB/s | 383 kB     00:00    
(167/221): nss-softokn-devel-3.59.0-2.fc34.x86_ 332 kB/s |  16 kB     00:00    
(169/221): nss-softokn-freebl-devel-3.59.0-2.fc 438 kB/s |  64 kB     00:00    
(170/221): nss-softokn-freebl-3.59.0-2.fc34.x86 2.1 MB/s | 329 kB     00:00    
(171/221): nss-sysinit-3.59.0-2.fc34.x86_64.rpm 621 kB/s |  21 kB     00:00    
(172/221): nss-util-3.59.0-2.fc34.x86_64.rpm    2.1 MB/s |  91 kB     00:00    
(173/221): nss-tools-3.59.0-2.fc34.x86_64.rpm    10 MB/s | 532 kB     00:00    
(174/221): nss-util-devel-3.59.0-2.fc34.x86_64. 1.9 MB/s |  77 kB     00:00  


So -2 build. :/

Comment 19 Bob Relyea 2020-12-17 23:20:42 UTC
Doh! the patch wasn't upstreamed yet
. I'll add it to nss-3.59 and rebuild.

bob

Comment 20 Ben Cotton 2021-11-04 16:19:09 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 21 Bob Relyea 2021-11-09 16:00:01 UTC
This should be fixed in the latest NSS.


Note You need to log in before you can comment on or make changes to this bug.