Hide Forgot
Created attachment 1216614 [details] test.c When porting perl-Crypt-OpenSSL-DSA to OpenSSL-1.1.0, I found an issue with failing DSA_generate_parameters_ex() call <https://github.com/kmx/perl-Crypt-OpenSSL-DSA/pull/3>. If attached code is built against openssl-1.1.0b-3.fc26.x86_64, it fails for some seeds lengths or values: [test@fedora-26 tmp]$ gcc -Wall test.c $(pkg-config --libs openssl) [test@fedora-26 tmp]$ ./a.out [test@fedora-26 tmp]$ ./a.out a Could not generate DSA parametetes: q not prime [test@fedora-26 tmp]$ ./a.out fo Could not generate DSA parametetes: q not prime [test@fedora-26 tmp]$ ./a.out foo [test@fedora-26 tmp]$ ./a.out fooo Could not generate DSA parametetes: q not prime [test@fedora-26 tmp]$ ./a.out foo [test@fedora-26 tmp]$ ./a.out abc Could not generate DSA parametetes: q not prime Why does it pass for "foo" seed but fails for "abc" seed? Why it fails for seeds with slight different length. Could it be caused by FIPS patch that uses dsa_builtin_paramgen2() instead dsa_builtin_paramgen()? The same code built against openssl-1.0.2j-1.fc24.x86_64 always works.
DSA_generate_parameters_ex(3) manual reads: bits is the length of the prime p to be generated. For lengths under 2048 bits, the length of q is 160 bits; for lengths greater than or equal to 2048 bits, the length of q is set to 256 bits. and Seed lengths > 20 are not supported. But it does not work even with exactly 20-bytes long seed that should be required for 512-bit p prime.
Not every seed generates a prime, however the old algorithm simply used random seed (or possibly multiple random seeds) if the seed failed to generate a prime. I will modify the dsa_builtin_paramen2 to behave the same way. I'll also modify the manual page to properly document the supported seed length as it actually depends on the SHA2 variant used internally. Anyway I would not bother testing the generation based on the pre-set seed at all.