Hide Forgot
crypto-utils should be ported to use NSS library for cryptography. See the tracking bug for details and links on how it could be done.
Created attachment 299417 [details] patch for certwatch and genkey crypto-utils crypto-utils.nss.patch:- diffs for certwatch.c, keygen.pl, certwatch.cron, the spec file and Makefile. pemutils.c: new, pem file support for certwatch keutil.c: new, keygen/certgen/csrgen/keywrap a la openssl, work in early stages of development
Created attachment 299418 [details] patch for certwatch and genkey crypto-utils crypto-utils.nss.patch:- diffs for certwatch.c, keygen.pl, certwatch.cron, the spec file and Makefile. pemutils.c: new, pem file support for certwatch keutil.c: new, keygen/certgen/csrgen/keywrap a la openssl, work in early stages of development
Excuse me, as the second one is an unintended repeat of the first one and the introduction didn't go out. This work, still in progress, is for your kind review. The goals are to have cert-utils use nss for the cryptography, support the nss way of handling certificates via the key and cert database for Apacahe configured with mod-nss, and maintain existing functionality for Apache configured with mod-ssl. The changes the cert watcher accomplish this I believe. The changes to genkey are still a work in progress specially in the area of openssl compatibility.
Created attachment 299429 [details] pem file support utilities. It didn't make it last time.
Created attachment 299430 [details] source diff This big diff file didn't make it last time.
Created attachment 299432 [details] The right diff file Finally, this the intended diff file. Please ignore previous one.
makeCert this call system("$bindir/openssl req -config $sslconf -new -key $keyfile $days -out $certfile < $tempfile 2> /dev/null"); Not everything is explicitly specified letting openssl look at the configuration file. According to documenation at http://www.openssl.org/docs/apps/req.html# the openssl req command may prompt the user for the relevant field values. The actual fields prompted for and their maximum and minimum sizes are specified in the configuration file and any requested extensions. I'm switching to using keytil which would have to do this parsing. It maybe easier to do the parsing and any user prompts here in the Perl script and pass everything needed to keyutil. What could the prompts be if someone changes the sslconf file? I have made a list of options to determine which ones may trigger parsing of the file. OPTION TO PARSE OR NOT, and WHY [-inform PEM|DER] -- No, always use PEM [-outform PEM|DER] -- No, always PEM [-in filename] -- No, we are using -new and -key [-passin arg] -- No, we don't have input file [-out filename] -- No, we are supying it here [-passout arg] -- No, will supply later when exporting encrypted key [-text] -- No, don't print the cert [-pubkey] -- No, we don't need to ouput the public key [-noout] -- No, we don't output the DER [-verify] -- No, always verify [-modulus] -- not used, we don't print it [-new] -- No, we are always making a new one [-rand file(s)] -- No, its specified here [-newkey rsa:bits] -- No, its already specied [-newkey dsa:file] -- No, dsa not supported [-newkey alg:file] -- No, not used [-nodes] -- No, key was generated [-key filename] -- No, we pass it here [-keyform PEM|DER] -- No, its always PEM, won't let sslconf [-keyout filename] -- No, we pass it here [-[md5|sha1|md2|mdc2]]-- No, use md5 [-config filename] -- No, it's specified here [-subj arg] -- No, we pass that here [-multivalue-rdn] -- No, could come from the config --- NO SURE !!! [-x509] -- No, not needed by keyutil [-days n] -- No, specified here (sometimes empty) [-set_serial n] -- not here, maybe in config ? [-asn1-kludge] -- No [-extensions section] -- Maybe [-reqexts section] -- Maybe, Do we need to support this? [-utf8] [-nameopt] -- No, but may be needed for internationalization [-batch] -- non-interactive mode - highly desirable as long can get all the information [-verbose] -- maybe, useful for debugging [-engine id] -- No, not applicable Please comment on my interpretation this list.
Per some off-line discussion - there's no need to try to maintain compatibility with all the various different ways that the OpenSSL configuration can be tweaked. Just use sane defaults :)
If the user chooses to encrypt the private key genkey calls openssl as follows: ... $bindir/openssl rsa -des3 -in $keyfile -passout stdin -out $enckey ... The docs at http://www.openssl.org/docs/apps/rsa.html# state: "The *rsa* command processes RSA keys. They can be converted between various forms and their components printed out. Note* this command uses the traditional SSLeay compatible format for private key encryption: newer applications should use the more secure PKCS#8 format using the *pkcs8* utility." genkey is a new application, isn't it? Was there a compelling reason why the ssleay way was used rather than pkcs8 as openssl recommends for new apps? NSS to my knowledge doesn't support the old ssleay way of doing things. It would be a major effort and less secure. This is what I see in NSS secoidt.h These 3 SEC_OID_PKCS5_PBE_WITH_MD2_AND_DES_CBC = 21, SEC_OID_PKCS5_PBE_WITH_MD5_AND_DES_CBC = 22, SEC_OID_PKCS5_PBE_WITH_SHA1_AND_DES_CBC = 23, are not PKCS #5 V2 and among these /* SEC_OID_PKCS12_OFFLINE_TRANSPORT_MODE, SEC_OID_PKCS12_ONLINE_TRANSPORT_MODE, */ .... SEC_OID_PKCS12_PKCS8_KEY_SHROUDING = 109, SEC_OID_PKCS12_PBE_WITH_SHA1_AND_TRIPLE_DES_CBC = 117, ... /* end of PKCS 12 additions */ SEC_OID_PKCS12_PKCS8_KEY_SHROUDING may be compatible with the pkcs8 command in openssl if we go that way. Please offer guidance.
SEC_OID_PKCS12_PBE_WITH_SHA1_AND_TRIPLE_DES_CBC Will give you a PKCS 8 encoded with the pkcs 12 v2 PBE (format similiar to pkcs 5 v1). If you just pass SEC_OID_TRIPLE_DES+_CBC you should get pkcs 8 encoded with pkcs 5 v2. I think the former is currently more common, but either should be understandable by openssl. bob
We double-checked and SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC is the one to use. Let me outline the approach taken to export the key. There will be no separate generation key or encrypt key steps. Key generation and export happen as part of the genreq (or makeca) phases. At this time we have access to the key and can export it to a file. The keyutil app will get at the key via the PK11_ExportEncryptedPrivKeyInfo call and transform the structure to DER encoding and 64 encode that. If an encrypted key is desired we just write it out to a file PEM encoded. The clear text key request turns out to be more difficult as NSS doesn't support exporting unencrypted private keys. While we have access to the SECKEYEncryptedPrivateKeyInfo struct we'll have to decrypt it massage before we can write it out PEM encoded. I'm using the openssl pkcs8 and pkcs12 commands to test. As I write the test aren't passing yet, I'm still debugging the "easy" part.
I've had a look into this. Whilst OpenSSL does support PKCS#8, the functions mod_ssl uses to read private keys will only accept the native OpenSSL format. It is a simple change to fix this in mod_ssl, though. So if you've tested that "openssl pkcs8" can read the keys produced by NSS in that format, this looks fine to me; I'll get the necessary changes committed to mod_ssl.
Ah, actually I tested that wrong. mod_ssl groks *PEM-encoded* PKCS#8 key files fine. (just not raw DER)
Actually, SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC was the wrong choice as my test revealed. openssl pkcs8 didn't like it. Then I used the original genkey to spit out a key, converted this sslay-style key to the modern format and the nss pp tool told me is encrytped DES_EDE_3. This means I need to export it using PKCS #5 v2 PCE with sha1 and Triple DES 3 key. Support for this came in NSS 3.11.9, what 3.12 is based om and available on fedora 9.
The openssl pkcs8 command is able decrypts the key that genkey (using keyutil) exports. The next feature is to export an unencrypted key.
Created attachment 304057 [details] diff file for existing higher level sources Until my cvs commit access gets approved I though I'd share another preview this way. The new source files will be included separately and also more updates when clear key export is completed. Some may want to view the genkey script in the meantime.
Created attachment 304280 [details] source diff for higher level sources All functionality except clear key export. New source files in subsequent attachments attachments. cvs checkins coming soon to devel branch.
Created attachment 304281 [details] nspr related internal header Maps nspr error codes to informative error strings.
Created attachment 304282 [details] sec eror realted internal header Maps sec error codes to informative error strings.
Created attachment 304284 [details] error messages programmer friendly error messages
Created attachment 304285 [details] security utilities header
Created attachment 304286 [details] security utilities This is code from a static library that nss utilities and test use. Cant' bring an nss internal, or any other, static library into fedora so I extracted the subset that our tool uses.
Created attachment 304287 [details] pem realted code for certwatch Duplicates some secutil functionality but we want certwatch small. This may find its way into nss-compat-ossl in the future.
Created attachment 304288 [details] keyutil support export of keys For compatibility with OpenSSL we must exprt keys along with the certs. keyutil does it. It's based on certutil.
Created attachment 304289 [details] internal header
Created attachment 304290 [details] supprt for certificate extensions This may not be needed after all.
I'm know checking in sources into the devel branch. Adding here some notes to help in code review. genkey, when running on nss mode, will dynamically gather some configuration information from the nss.conf file, namely the location of the database, the server nickname, and the database prefix if any. This parameters will be passed down to certutil. The cron job will do similarly for its certwatch application. The nss.conf has only one entry for NSSDBPrefix so I assume it's used for both the cert and key prefix if at all. The location of nss.conf is determined dynamically. If it isn't in /etc/httpd/conf.d/nss.conf then we do an rpm query to find it and parse for the needed information.
Created attachment 305047 [details] diff current sources against F-9-split tag Source diff local sources against the F-9-split tag for review.
Created attachment 305048 [details] source rpm crypto-utils ported to nss source rpm crypto-utils ported to nss for review.
Expanding on comment #28. genkey at runtime determines, via rpm -q, whether mod_nss or mod_ssl is installed and it advises the user that one of them must be present in order to either generate a CSR or CA self-signed cert for the corresponding configuration of Apache. In the spec file neither mod_ssl nor mod_nss appear in the BuildRquires nor in the Requires section as we cannot assume either one is installed ahead of time. Some installations will have both and others one and not the other.
Attached is a source rpm and a diff file (against the F-9-split tag) for your review.
The genkey tool documentation does not explicitly state that it can be used for renewing certificates but when one consults the OpenSSL req command documentation and examines the way its invoked by the shipping genkey it could be used to renew a certificate. The submitted for review doesn't not support certificate renewal, CSR generation always generates a fresh key pair. Genkey using openssl has a separate key generation step whereas the one with nss does the key pair generation as part of the CSR (or cert) generation steps. When creating a CSR (the genreq_mode) old genkey passes to openssl an existing key whereas the ported one doesn't. Let's look art current practice first. For both cert and CSR generation genkey has the overloaded makeCert subroutine that invokes openssl with: system("$bindir/openssl req -config $sslconf -new -key $keyfile $days -out $certfile < $tempfile 2> /dev/null"); Here $tempfile has the certificate details so the user doesn't get prompted for them. $keyfile is an existing private key. It's usually one made when genkey was invoked in test mode. The user can make this file be the existing private key file for a cert that has expired. That is what I was refer by renewal. The documentation is not explicit about it but a user in the know, who got a cert expiry email from certwatch, may have that expectation and may want to reuse the key pair. The nss tools documentation doesn't explicitly address certificate renewal either. Renewal is addressed in the certificate system documentation with an example that's now deprecated. We had an off-line chat on the subject. Rob pointed us to https://bugzilla.mozilla.org/show_bug.cgi?id=341371 The thread is lengthy but what it boils down is that you can do a renewal by using the nickname as the "pointer" to what cert you want renewed in the cert util -R invocation CSR "regular" usage looks like: certutil -R -s subject -k rsa -g bits \ #<----- key type and key size -d dir -o csrfile -t trust-args .. other arguments ommitted whereas for a renewal: certutil -R -s $subject -k nickname \ #<---- instead of rsa use nickname -y publicExponent -t trust_args -d dir -o outcsrfile ... other arguments The -k option is actually overloaded! We would need to extract the subject, trust, publicExponent from the database. The certutil -L (list) call provides the info in human-friendly format and a bit of text processing is needed to extract the pieces in usable form This only addresses the case where the keys/certs are in the database. The OpenSSL renewal would be more difficult. Any ideas? But first, is supporting reuse of keys a requirement?
Created attachment 308356 [details] diff 2.4-2 against F-9-split tag Actual sources built by Koji.
Regarding comment #32: neither the -y publicExpoenet nor -t trust_args is needed and certutil shouldn't require the subject either when doing renewals. Extracting the subject by parsing the output of a certutil -L -d dir -n nickname is tricky as sometimes it spills over into a second line. From the nickname certutil can get at the cert and from it get at the subject. I have logged a bug upstream: Bug #437804 Certutil -R for cert renewal should derive the subject from the cert if none is specified, see https://bugzilla.mozilla.org/show_bug.cgi?id=437804
The certutil patch submitted to mozilla was accepted, the code is checked in, and come NSS 3.12.2 certutil will do what we want. That should enable adding to genkey support for true certificate renewal when using the nss database. The OpenSSL renewal case turns out not be as difficult as I thought earlier. Genkey can use the new PEM PKCS #11 module that was added to fedora for this case. The PEM module needs some additional work. We may also need to rely on the user to indicate whether the renewal is for a server or a CA certificate. I should be able to revisit this issues again.
From an email exhange: Elio Maldonado wrote: Joe, The crypto-utils work to use NSS I did is not complete. To finish I must wait for a patch I did to certutil upstream which will not become available until NSS 3.12.2 comes out and it gets picked up downstream in Fedora. It also needs further review and testing. I certainly don't want the crypto-utils changes to be included in the upcoming Fedora 10 release. How do I ensure that it doesn't get moved to Fedora-10? Joe Orton wrote: So, the status quo is that "genkey --genreq" will work with an existing private key in the NSS database, but not in a PEM file on disk? I don't think that's a showstopper. There is presumably a (manual) way to import the PEM private key into the NSS database anyway? Joe, genkey handles PEM files just fine as I take avantage of the PEM module Rob Crittenden wrote. The problem is renewing certs from the nss database (reusing the key as opposed to making a CSR for a new cert). The fix we depend on is upstream but NSS 3.12.2 won't ship until after Fedora 10 ships.
OK. Can we ship updates for NSS & crypto-utils after F10 is released, which fix this?
As far as I know Kai patches Fedora NSS every time an official NSS release comes out from upstream. At that time, or shortly after, I could update crypto-utils to add the missing functionality. Is that your proposal?
Is the missing patch to NSS the ONLY remaining problem with porting crypto-utils to NSS? I was under the impression that Elio sees more problems, and thinks it's not yet ready. (I may have misunderstood). If the patch for certutil is the ONLY thing you are waiting for, to get this working correctly, I'm OK to get this patch into our NSS RPM earlier than the release of NSS 3.12.2 In any way, NSS 3.12.2 will certainly get shipped as an update to F10 once its available.
Your impression is correct. I do see more problems in addition to the missing patch. For example, the part that renews certs for certs and keys works for server certs but not for ca certs it needs work as I mentioned previously in comment #35. The previous reviews were only partial reviews and done many months ago, genkey.pl hasn't been reviewed at all, plus I have made changes since then. This is not a minor patch, it's a major change to crypto-utils and needs a complete and thorough review before we can claim to be ready for inclusion fedora 10. I do need to resubmit it for review but I won't have time until next week.
Created attachment 320816 [details] OpenSSL style cert request and generation, inspired on certutil and libcurl. Called by genkey when in OpenSSL compatibility mode. Uses the PEM PKCS #11 module to load keys/certs from PEM files on the fly. Loading code was adapted from that in the nss-enabled libcurl (i.e. nss.c).
Created attachment 320817 [details] header for keyutil
Created attachment 320818 [details] Openssl compatibilty for use by certwatch
Created attachment 320819 [details] error reporting support Adapted from the internal library used by nss security tools.
Created attachment 320820 [details] export function for adding extensions
Created attachment 320821 [details] security utilities from nss security tools
Created attachment 320822 [details] header for security utilities
Created attachment 320823 [details] changes since the F-9-split tag For Joe Orton's review.
I've done a brief review of the genkey.pl changes and they look fine. If it's correct that the the lost functionality can be restored as an F10 update, and you've performed reasonably thorough testing then I'm fine with going ahead with this for F10.
Created attachment 321049 [details] changes to add support certificate renewal
Created attachment 321051 [details] changes to support cert renewal from pem files
For Bob Relyea's review: thechanges for cert renewal support are since the last review. Keyutil calls the PEM PKCS #11 module with code heavily "inspired" on Rob Crittenden's code in libcurl. Caveat, I am still testing it.
A new rawhide build to bring in the desired dependent certutil patch is underway: http://koji.fedoraproject.org/koji/buildinfo?buildID=67055
Created attachment 321681 [details] security utilities exports
Created attachment 321682 [details] openssl style cert signing request and generation
Created attachment 321683 [details] changes since F-9-split
Created attachment 321684 [details] changes since 2.4-2, last review
This work to use NSS actually shipped in F-9, had a couple of fixes on F-10, will certainly get more tender care in Rawhide for F-12. How come we never updated the status?
I bet this is so is because crypto-utils is also a RHEL package and has been for a while. It does make sense.
Since crypto-utils does work with NSS now, closing out.