Bug 748394 - SEC_ERROR_BAD_SIGNATURE with a certificate trusted by OpenSSL
SEC_ERROR_BAD_SIGNATURE with a certificate trusted by OpenSSL
Product: Fedora
Classification: Fedora
Component: nss (Show other bugs)
Unspecified Unspecified
medium Severity high
: ---
: ---
Assigned To: Elio Maldonado Batiz
Fedora Extras Quality Assurance
Depends On:
Blocks: 748401
  Show dependency treegraph
Reported: 2011-10-24 06:49 EDT by Jan Vcelak
Modified: 2013-03-03 20:29 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 748401 (view as bug list)
Last Closed: 2012-08-07 15:18:47 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
full backtrace (9.77 KB, text/plain)
2011-10-24 06:50 EDT, Jan Vcelak
no flags Details
data for reproduction (5.08 KB, application/x-compressed-tar)
2011-10-24 07:06 EDT, Jan Vcelak
no flags Details

  None (edit)
Description Jan Vcelak 2011-10-24 06:49:39 EDT
Description of problem:

The problem can be reproduced with OpenLDAP. It was reported in OpenLDAP upstream issue tracker (http://www.openldap.org/lists/openldap-bugs/201110/msg00021.html). In fact this is a problem of Mozilla NSS. Fedora is affected.

Version-Release number of selected component (if applicable):


Steps to Reproduce:
1. Create CA certificate with DSA signature
2. Create server certificate with RSA signature
3. Sign the certificate by generated CA certificate
4. setup slapd to use these certificates (TLSCertificateFile, TLSCertificateKeyFile, do not set TLSCACertificateFile) and start the server
5. LDAPTLS_CACERT=/your/cacert.pem ldapsearch -x -ZZ -d1 -H ldap://your-server

Actual results:

TLS: loaded CA certificate file /tmp/CA/CA/cacert.pem.
TLS: certificate [CN=alioth.usersys.redhat.com,O=jvcelak Red Hat Test,L=Brno,C=CZ] is not valid - error -8182:Unknown code ___f 10.
TLS: error: connect - force handshake failure: errno 21 - moznss error -8182
TLS: can't connect: TLS error -8182:Unknown code ___f 10.
ldap_start_tls: Connect error (-11)
        additional info: TLS error -8182:Unknown code ___f 10

The server side result:

connection_get(12): got connid=1002
connection_read(12): checking for input on id=1002
connection_get(12): got connid=1002
connection_read(12): checking for input on id=1002
TLS: error: accept - force handshake failure: errno 11 - moznss error -12271
TLS: can't accept: TLS error -12271:Unknown code ___P 17.
connection_read(12): TLS accept failure error=-1 id=1002, closing

Expected results:

The connection will succeed.

Additional info:

The failure comes from DSAU_ConvertSignedToFixedUnsigned ->  line 120.

116     if (zCount <= 0) {
117         /* Source is longer than destination.  Check for leading zeros. */
118         while (zCount++ < 0) {
119             if (*pSrc++ != 0)
120                 goto loser;
121         }
122     }

I will attach full backtrace. And configuration file with certificates.

(Do not set TLSCACertificateFile in step 4., otherwise the server certificate validation will fail on the server side with -8182 and all following TLS requests will be refused. This configuration is easier for debugging, you do not have to restart the server with every request.)
Comment 1 Jan Vcelak 2011-10-24 06:50:16 EDT
Created attachment 529788 [details]
full backtrace
Comment 2 Jan Vcelak 2011-10-24 07:06:16 EDT
Created attachment 529827 [details]
data for reproduction

Extract the archive into /tmp.

start the server: /tmp/slapd-dsa-rsa/run-server.sh
query the server: /tmp/slapd-dsa-rsa/query-server.sh
Comment 3 Fedora End Of Life 2012-08-07 15:18:50 EDT
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:

Note You need to log in before you can comment on or make changes to this bug.