product: e_nss - An OpenSSL engine for Network Security Services (NSS) home: https://roumenpetrov.info.example.net/e_nss/ repo: https://gitlab.com/e_nss/e_nss/ e_nss contains a number of regression tests. All those tests pass builds with various openssl releases (0.9.7 - 1.1.1) and nss releases (tested up to 3.73). Tests includes OpenSSL sign and verify operations with keys provided by NSS. Recently e_nss start to fail on sign operation with dsa key. First was visible on $ cat /etc/centos-release CentOS Linux release 8.5.2111 with nss 3.67.0-7.el8_5, openssl 1.1.1k-5.el8_5 and compiler from subject.
More information: 1) missing debug package? # debuginfo-install nss ... Could not find debuginfo package for the following installed packages: nss-3.67.0-7.el8_5.x86_64 Could not find debugsource package for the following installed packages: nss-3.67.0-7.el8_5.x86_64 $ cat /etc/centos-release CentOS Linux release 8.5.2111 nss.x86_64 3.67.0-7.el8_5 openssl.x86_64 1:1.1.1k-5.el8_5 2) e_nss regression tests pass on all nss releases up to 3.73. Remark 3.73.1 is not tested yet. In particular nss-3.67-with-nspr-4.29.tar.gz and distibution is not redhat based. 3) failed test is tests/sign-dsa-dss1.test, error is SEC_ERROR_SIGNATURE_ALGORITHM_DISABLED returned from method SGN_Digest(). 4) there is no indication that custom patches in nss-3.67.0-7.el8_5.src.rpm stops dsa keys. 5) test fail even with NSS_IGNORE_SYSTEM_POLICY=1 Remark: system policy disables dsa keys less then 2048, test intentionally uses 1024 and environment variable should avoid system limitatiosn. 6) all test pass if build is on another distibutuon with all paches from nss-3.67.0-7.el8_5.src.rpm 7) non optimised rpmbuild fail 7.1) all custom paches commented 7.2) # Enable compiler optimizations and disable debugging code #export BUILD_OPT=1 7.3) # Uncomment to disable optimizations RPM_OPT_FLAGS=`echo $RPM_OPT_FLAGS | sed -e 's/-O2/-O0/g'` export RPM_OPT_FLAGS The reason is that this is forty source enabled build: $ grep -i optim /usr/include/features.h # if !defined __OPTIMIZE__ || __OPTIMIZE__ <= 0 # warning _FORTIFY_SOURCE requires compiling with optimization (-O) #if __GNUC_PREREQ (2, 7) && defined __OPTIMIZE__ \ && !defined __OPTIMIZE_SIZE__ && !defined __NO_INLINE__ \ As result above line must be changed to: RPM_OPT_FLAGS=`echo $RPM_OPT_FLAGS | sed -e 's/-O2/-O0/g' -e 's/-Wp,-D_FORTIFY_SOURCE=2//g'` 8) even with non optimised nss dsa sing test fail
Finaly some debug details: (gdb) list 291 292 result.type = siBuffer; 293 result.data = NULL; 294 result.len = 0; 295 296 rv = SGN_Digest(keyctx->pvtkey, hashalg, &result, &digest); 297 nss_trace(ctx, "rv=%d\n", rv); 298 if (rv != SECSuccess) { 299 int port_err = PORT_GetError(); 300 switch(port_err) { (gdb) display 5: keyctx->pvtkey = (SECKEYPrivateKey *) 0x5555558c8b40 6: *keyctx->pvtkey = {arena = 0x55555588cf10, keyType = dsaKey, pkcs11Slot = 0x5555558863f0, pkcs11ID = 3297886637, pkcs11IsTemp = 0, wincx = 0x7fffffffd050, staticflags = 0} (gdb) s SGN_Digest (privKey=0x7ffff76f27a3 <ASN1_item_ex_i2d+691>, algtag=32767, result=0xffffffff, digest=0x0) at secsign.c:462 462 { (gdb) As is visible SGN_Digest is called with keyctx->pvtkey equal to 0x5555558c8b40 but in function parameter is changed to 0x7ffff76f27a3. Note that RSA and EC keys use similar code but failure is only for dsa keys. Now as work-around is used local variable: before: rv = SGN_Digest(keyctx->pvtkey, hashalg, &result, &digest); after: { SECKEYPrivateKey *pvtkey = keyctx->pvtkey; rv = SGN_Digest(pvtkey, hashalg, &result, &digest); }
Would it be possible to extract a stand alone test case that I can actually try to compile and run? Maybe the test is just a single .c file that could be preprocessed (-save-temps) and then attached here. Is there a gcc version with which the test passes? I'm not convinced that this is a gcc bug given that the problem manifests even without optimizations. There may be undefined behavior happening in the test, you could try -fno-strict-aliasing or compile/run the test with -fsanitize=undefined to see if it helps.
I did't check assembler before to post. It seems to me it is one and the same for DSA (subject of issue), RSA and EC around call of method SGN_Digest. With other words this report in invalid.
OK, thanks. Closing.