Bug 1551702

Summary: certmonger unit test fails with SEC_ERROR_BAD_DER
Product: Red Hat Enterprise Linux 7 Reporter: Rob Crittenden <rcritten>
Component: certmongerAssignee: Rob Crittenden <rcritten>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: urgent Docs Contact:
Priority: high    
Version: 7.5CC: dueno, hkario, kengert, ksiddiqu, lmiksik, mgrepl, mkosek, nalin, ndehadra, pvoborni, rcritten, rrelyea, salmy, toneata
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: certmonger-0.78.4-4.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1560960 (view as bug list) Environment:
Last Closed: 2018-10-30 07:44:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1551635, 1560960    
Attachments:
Description Flags
cert.nss.2048
none
raw DER encoding of cert.nss.2048
none
stack.txt
none
cert text dump
none
minimal reproducer
none
Daiki's suggested fix as a patch to certmonger none

Description Rob Crittenden 2018-03-05 19:09:33 UTC
Description of problem:
certmonger unit test 004-selfsign-dsa is failing:

Generating key.  This may take a few moments...

Notice: Trust flag u is set automatically if the private key is present.
2018-03-05 13:38:56 [24095] Found token 'NSS Certificate DB'.
2018-03-05 13:38:56 [24095] Located a certificate with the key's nickname ("keyi2048").
2018-03-05 13:38:56 [24095] Located its private key.
2018-03-05 13:38:56 [24095] Recovered public key from private key.
2018-03-05 13:38:56 [24098] Found token 'NSS Certificate DB'.
2018-03-05 13:38:56 [24098] Located a certificate with the key's nickname ("keyi2048").
2018-03-05 13:38:56 [24098] Located its private key.
2018-03-05 13:38:56 [24098] Recovered public key from private key.
2018-03-05 13:38:56 [24098] Setting certificate serial number "04".
2018-03-05 13:38:56 [24106] Setting certificate serial number "04".
error in verification: SEC_ERROR_BAD_DER
error in verification: SEC_ERROR_BAD_DER
Test complete.

The test works with RHEL 7.4 nss-3.28.4-15.el7_4.x86_64 and no certmonger code changed in the interm.

Version-Release number of selected component (if applicable):
nss-3.34.0-4.el7.x86_64

How reproducible:

Every time

Steps to Reproduce:
1. Try to build certmonger in RHEL 7.5

Additional info:

A way to debug is in-tree in the certmonger source is to modify run.sh to include something like:

tmpdir="/tmp/foo"
srcdir="/home/<yourname>/rpmbuild/BUILD/certmonger-0.78.4/tests"
toolsdir="/home/<yourname>/rpmbuild/BUILD/certmonger-0.78.4/tests/tools"

CERTMONGER_CONFIG_DIR=${srcdir}
export CERTMONGER_CONFIG_DIR

cd "$tmpdir"

Now you can execute run.sh directly.

I did this by executing rpmbuild --rebuild certmonger-0.78.4-3.el7.src.rpm and then switching to the BUILD directory after the tests fail.

It also fails in 004-selfsign-ec

Comment 8 Kai Engert (:kaie) (inactive account) 2018-03-05 20:41:05 UTC
Created attachment 1404489 [details]
cert.nss.2048

Comment 9 Kai Engert (:kaie) (inactive account) 2018-03-05 20:41:47 UTC
Created attachment 1404490 [details]
raw DER encoding of cert.nss.2048

Comment 10 Kai Engert (:kaie) (inactive account) 2018-03-05 20:42:37 UTC
Created attachment 1404491 [details]
stack.txt

Comment 11 Kai Engert (:kaie) (inactive account) 2018-03-05 20:45:20 UTC
Created attachment 1404504 [details]
cert text dump

Bob, Hubert, any idea what properties of this certificate could cause NSS 3.36 to fail decoding this certificate with BAD_DER, while NSS 3.28 had worked?

Well, I don't know if this encoding is valid. Another potential explanation could be that NSS 3.36 produced an invalid encoding.

Comment 12 Alicja Kario 2018-03-06 15:19:00 UTC
I've got no clue, it looks sane and openssl parses it without issue, but it complaints about not being able to get issuer, despite this cert looking self-signed:

$ openssl verify -CAfile cert.nss.2048 cert.nss.2048
CN = Babs Jensen
error 20 at 0 depth lookup: unable to get local issuer certificate
error cert.nss.2048: verification failed

maybe "X509v3 Subject Key Identifier" is calculated wrong?

Comment 14 Kai Engert (:kaie) (inactive account) 2018-03-06 15:24:24 UTC
This bug is seen with NSS 3.34. It might be worth repeating the test with the nss-softokn 3.36 packages that Daiki is currently working on. Although I don't have a lot of hope that it makes a difference.

Comment 19 Daiki Ueno 2018-03-07 08:13:57 UTC
Created attachment 1405176 [details]
minimal reproducer

OK, I realized that this is a problem in the test program (it worked before because the old decoder is more tolerant about the length).  When decoding BIT_STRING either with the old decoder or QuickDER, the length is returned in the number of bits instead of octets; so 504 / 8 = 64.

On the other hand, VFY_VerifyDataWithAlgorithmIDI() expects octet length for the signature value.  So you need to adjust it with something like:

  signed_data.signature.len /= 8;

before calling that function.

I am attaching the standalone version of checksig.c, which can be compiled with:

  $ gcc -o checksig checksig.c `pkg-config nss --cflags --libs`

Feed the "der" file attached at comment 9:

  $ cat der | ./checksig
  error in verification: SEC_ERROR_BAD_DER

If you comment out the line 95 (which adds the above adjustment) and recompile it:

  $ cat der | ./checksig
  verification OK

Comment 20 Kai Engert (:kaie) (inactive account) 2018-03-07 10:22:00 UTC
Bob, thanks for the analysis.
Daiki, thanks for identifying the fix!

I'm attaching Daiki's suggested fix, and reassigning to certmonger.

Rob, could you please verify it fixes the issue?

Comment 21 Kai Engert (:kaie) (inactive account) 2018-03-07 10:22:47 UTC
Created attachment 1405225 [details]
Daiki's suggested fix as a patch to certmonger

Comment 25 Rob Crittenden 2018-03-07 15:17:49 UTC
The patch works for me, thank you very much! It doesn't look like any changes to NSS are required.

Comment 31 Nikhil Dehadrai 2018-08-03 11:41:45 UTC
IPA version: ipa-server-4.6.4-3.el7.x86_64
Certmonger: certmonger-0.78.4-3.el7.x86_64

Verified that the sanity tests required for verifying this bug are run successfully.

No errors are observed when test run for ipa-getcert are run as regression for latest version.

Thus marking the status of bug to Verified, based on these observations.

Comment 33 Nikhil Dehadrai 2018-08-03 11:46:44 UTC
(In reply to Nikhil Dehadrai from comment #31)
> IPA version: ipa-server-4.6.4-3.el7.x86_64
> Certmonger: certmonger-0.78.4-3.el7.x86_64
> 
> Verified that the sanity tests required for verifying this bug are run
> successfully.
> 
> No errors are observed when test run for ipa-getcert are run as regression
> for latest version.
> 
> Thus marking the status of bug to Verified, based on these observations.

certmonger version: certmonger-0.78.4-8.el7.x86_64

Comment 35 errata-xmlrpc 2018-10-30 07:44:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3018