Bug 1764205

Please add reproduction steps how to create PKCS#12 files in Firefox.

about:preferences#privacy
Very bottom of page, Certificates section click "View Certificates"
Go to "Your Certficates" tab, select a certificate, click Backup...

Bob, could you please help me with the correct openssl->nss crypt types translation, I've naively replaced the:
SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_40_BIT_RC2_CBC by SEC_OID_AES_128_CBC
and
SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC by SEC_OID_SHA256 in the https://searchfox.org/mozilla-central/source/security/manager/ssl/nsPKCS12Blob.cpp#109
(btw it didn't work).

I'm looking at https://searchfox.org/mozilla-central/source/security/nss/lib/util/secoidt.h#221 and happen to be quite puzzled which SECOidTag to use in the Firefox.

This is what Hubert is expecting:
https://bugzilla.redhat.com/show_bug.cgi?id=1759982#c0

I'm not familiar with NSS on the API level, but there's a big difference between the RC4 or the SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC algorithms and the AES-128-CBC encryption: the former are PBES1 while the latter is PBES2 (and thus uses PBKDF2), so it's likely that they will require a different API call to achieve AES-128-CBC encryption.

I can suggest looking into pk12util, and checking how it handles "AES-128-CBC" for -c and -C options.

Sorry I can't help more.

Firfox is likely using the old pkcs5v1 interface. For compatibility, we recognized the SEC_OID_AES_XXX_CBC, and automatically encoded it  with SEC_OID_HMAC_SHA1.

I'd replace SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_40_BIT_RC2_CBC with SEC_OID_AES_128_CBC and SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC with SEC_OID_AES_256_CBC. We can handle the SHA1 issue in NSS in a separate bug.

NSS has separate API's at the lower level to do PBE2 and specify the PRF, but that isn't exported through the PKCS #12 interface unfortunately.

Looks like the same issue exists for pk12util in NSS as well.

bob

I managed to create a backup of the key with new enums, but the output may have some flaws:

> openssl pkcs12 -passin pass: -passout pass: -in backup.p12 -out /dev/null -info -noout
> MAC: sha1, Iteration 600000
> MAC length: 20, salt length: 16
> PKCS7 Data
> Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 600000, PRF hmacWithSHA1
> PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA1
> Error outputting keys and certificates
> 140432681781056:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:crypto/evp/evp_enc.c:616:
> 140432681781056:error:23077074:PKCS12 routines:PKCS12_pbe_crypt:pkcs12 cipherfinal error:crypto/pkcs12/p12_decr.c:62:
> 140432681781056:error:2306A075:PKCS12 routines:PKCS12_item_decrypt_d2i:pkcs12 pbe crypt error:crypto/pkcs12/p12_decr.c:93:


From the bug 1759982 we should get something like: 
> MAC: sha256, Iteration 600000
> MAC length: 32, salt length: 8
> PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA256
> Certificate bag
> PKCS7 Data
> Shrouded Keybag: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA256

SHA1/hmacWithSHA1 seems to be explained already (do you want me to fill the bug for it for the nss?). For the AES-256-CBC, I can use the AES-128-CBC to have the same output as bug 1759982 has. But there are some errors. Maybe the implementation between nss and openssl differs in the backup file format?

For the current implementation (without any changes to the Firefox), I get: 
> MAC: sha1, Iteration 600000
> MAC length: 20, salt length: 16
> PKCS7 Data
> Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 600000
> PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 600000
> Certificate bag
> Certificate bag

I can attach the p12 file if it helps.

(In reply to Jan Horak from comment #7)
> I managed to create a backup of the key with new enums, but the output may
> have some flaws:
> 
> > openssl pkcs12 -passin pass: -passout pass: -in backup.p12 -out /dev/null -info -noout
> > MAC: sha1, Iteration 600000
> > MAC length: 20, salt length: 16
> > PKCS7 Data
> > Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 600000, PRF hmacWithSHA1
> > PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA1
> > Error outputting keys and certificates
> > 140432681781056:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:crypto/evp/evp_enc.c:616:
> > 140432681781056:error:23077074:PKCS12 routines:PKCS12_pbe_crypt:pkcs12 cipherfinal error:crypto/pkcs12/p12_decr.c:62:
> > 140432681781056:error:2306A075:PKCS12 routines:PKCS12_item_decrypt_d2i:pkcs12 pbe crypt error:crypto/pkcs12/p12_decr.c:93:

that's bad, this suggests that the decryption key is incorrect and the padding check failed

> From the bug 1759982 we should get something like: 
> > MAC: sha256, Iteration 600000
> > MAC length: 32, salt length: 8
> > PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA256
> > Certificate bag
> > PKCS7 Data
> > Shrouded Keybag: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA256
> 
> SHA1/hmacWithSHA1 seems to be explained already (do you want me to fill the
> bug for it for the nss?). For the AES-256-CBC, I can use the AES-128-CBC to
> have the same output as bug 1759982 has.

the important part is AES, the specific key size is secondary

> But there are some errors. Maybe
> the implementation between nss and openssl differs in the backup file format?

it shouldn't, we already have an extensive interoperability test suite between
NSS, GnuTLS, and OpenSSL; all of them should work with each-other
so that looks like an issue with the NSS API allowing creation of broken PKCS#12 files...
 
> For the current implementation (without any changes to the Firefox), I get: 
> > MAC: sha1, Iteration 600000
> > MAC length: 20, salt length: 16
> > PKCS7 Data
> > Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 600000
> > PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 600000
> > Certificate bag
> > Certificate bag
> 
> I can attach the p12 file if it helps.

there may be a mixup with the expected key size, yes, please send it, I'll check
if the generated ASN.1 is as expected

What happens if you just change the triple DES value?

There are two bags that we use in PKCS #12: one to encrypt the key and one to encrypt the certs. The certs have always been 'lightly' encrypted if at all.The Shrouded Keybag is the most important. The other option is to not encrypt the certs at all (that's what pk12util does in FIPS mode because NIST would rather no encryption than encryption with a week algorithm). You can accomplish that with SEC_OID_UNKNOWN.

bob

Created attachment 1756354 [details]
generated p12 file by using SEC_OID_AES_128_CBC and SEC_OID_AES_256_CBC

(In reply to Bob Relyea from comment #9)
> What happens if you just change the triple DES value?
> 
> There are two bags that we use in PKCS #12: one to encrypt the key and one
> to encrypt the certs. The certs have always been 'lightly' encrypted if at
> all.The Shrouded Keybag is the most important. The other option is to not
> encrypt the certs at all (that's what pk12util does in FIPS mode because
> NIST would rather no encryption than encryption with a week algorithm). You
> can accomplish that with SEC_OID_UNKNOWN.
> 
> bob

Without 3DES change I get:

MAC: sha1, Iteration 600000
MAC length: 20, salt length: 16
PKCS7 Data
Shrouded Keybag: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA1
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 600000
Certificate bag
Certificate bag

(In reply to Jan Horak from comment #10)
> Created attachment 1756354 [details]
> generated p12 file by using SEC_OID_AES_128_CBC and SEC_OID_AES_256_CBC

Use: openssl pkcs12 -passin pass:mojeheslo -in backup.p12 -out /dev/null -info -noout

the password is no secret.

weird, I can't open that file with openssl, but if I use pk12util to create a similar one:
```
$ pk12util -o /tmp/export.p12 -n server -d sql:/tmp/nssdb -c AES-256-CBC -C AES-128-CBC -W simplepass
pk12util: PKCS12 EXPORT SUCCESSFUL
```
it works just fine:
```
$ openssl pkcs12 -passin pass:simplepass -passout pass: -in /tmp/export.p12 -out /dev/null -info -noout
MAC: sha1, Iteration 600000
MAC length: 20, salt length: 16
PKCS7 Data
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 600000, PRF hmacWithSHA1
PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA1
Certificate bag
Certificate bag
```

the PBKDF2 settings in the PKCS#12 files are also correct (ask for 16 byte key for AES-128 and 32 byte key for AES-256)

So the pkcs12 file you supplied encodes the password as UCS2. In pkcs12, that was required, but pkcs5v2 it's supposed to be UTF8. It looks like NSS tries both on decrypt which is why NSS can import it. Openssl pkcs12 app can't handle UCS2 as inputs (it can only handle null terminated strings). The mac codes are always encoded in UCS2, which is why the mac works. I set a breakpoint in the openssl tool and manually set the encryption password to the UCS2 encoding (the openssl low level api's can handle that), and it worked.

The NSS command line pk12util tries to import a file without encoding, and if the fails, it switches to importing with UCS2. You can see this code in  P12U_ImportPKCS12Object in nss/cmd/pk12util (it might be a good idea for firefox to pick up this code if we find firefox can't import some p12 files after we make this fix).

Now the question is why is the password encoded in UCS2. The pkcs12 interface takes UTF8. Under the covers it converts UTF8 to UCS2 as needed. This happens correctly since NSS 3.31, so if your code was running on RHEL 9 (or even RHEL-7.9 or RHEL 8.3) and you don't have your own private copy of NSS, this should be working correctly.

Now the UTF8 to UCS2 conversion uses a callback routine (which firefox supplies, IIRC. pk12util explicitly selects the NSS internal conversion routine). NIf the callback routine recognizes UCS2 input and just returns it with no conversion when asked to convert UTF8 into UCS, but the UTF8 base data is already UCS2 and firefox supplies the password in UCS2 format itself, then we could get the behavior we are seeing.

In that case I recommend: 1) making sure the we only pass UTF8 data to the nss pk12 interface. If the input data is UCS2, convert it to UTF8. 2) register a UTF8 to UCS2 with NSS. The NSS interface was originally an ASCII to UCS2 encoder. If you do the full UTF8 to ucs2, then you'll be able to maintain full fidelity with your UCS2 input. I imagine firefox already has such a decoder internally. If not NSS supplies implementations in SEC_PORT_

bob

(In reply to Bob Relyea from comment #14)
> So the pkcs12 file you supplied encodes the password as UCS2. In pkcs12,
> that was required, but pkcs5v2 it's supposed to be UTF8. It looks like NSS
> tries both on decrypt which is why NSS can import it. Openssl pkcs12 app
> can't handle UCS2 as inputs (it can only handle null terminated strings).
> The mac codes are always encoded in UCS2, which is why the mac works. I set
> a breakpoint in the openssl tool and manually set the encryption password to
> the UCS2 encoding (the openssl low level api's can handle that), and it
> worked.
> 
> The NSS command line pk12util tries to import a file without encoding, and
> if the fails, it switches to importing with UCS2. You can see this code in 
> P12U_ImportPKCS12Object in nss/cmd/pk12util (it might be a good idea for
> firefox to pick up this code if we find firefox can't import some p12 files
> after we make this fix).
> 
> Now the question is why is the password encoded in UCS2. The pkcs12
> interface takes UTF8. Under the covers it converts UTF8 to UCS2 as needed.
> This happens correctly since NSS 3.31, so if your code was running on RHEL 9
> (or even RHEL-7.9 or RHEL 8.3) and you don't have your own private copy of
> NSS, this should be working correctly.
I've used the upstream in-tree nss (on mozilla-central), now with Fedora 32 (nss-3.60.1-1.fc32.x86_64). Basically I'm getting same result.


> Now the UTF8 to UCS2 conversion uses a callback routine (which firefox
> supplies, IIRC. pk12util explicitly selects the NSS internal conversion
> routine). NIf the callback routine recognizes UCS2 input and just returns it
> with no conversion when asked to convert UTF8 into UCS, but the UTF8 base
> data is already UCS2 and firefox supplies the password in UCS2 format
> itself, then we could get the behavior we are seeing.
> 
> In that case I recommend: 1) making sure the we only pass UTF8 data to the
> nss pk12 interface. If the input data is UCS2, convert it to UTF8. 
I'm not sure which pk12 interface consumes utf8. Looking at SEC_PKCS12CreatePasswordPrivSafe [1]
it rather use sec_pkcs12_encode_password [2] which for SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_40_BIT_RC2_CBC use sec_pkcs12_convert_item_to_unicode to "convert" to char16_t (basically seems to do nothing with under little endian arch).

(gdb) p *uniPwitem->data@25
$36 = "\000m\000o\000j\000e\000h\000e\000s\000l\000o\000\000\000\000\000\000"
(gdb) p *pwitem->data@25
$37 = "\000m\000o\000j\000e\000h\000e\000s\000l\000o\000\000\345\345\345\345", <incomplete sequence \345>

For the SEC_OID_AES_128_CBC the sec_pkcs12_convert_item_to_unicode seems to just copy the pwitem data.

> 2)
> register a UTF8 to UCS2 with NSS. The NSS interface was originally an ASCII
> to UCS2 encoder. If you do the full UTF8 to ucs2, then you'll be able to
> maintain full fidelity with your UCS2 input. I imagine firefox already has
> such a decoder internally. If not NSS supplies implementations in SEC_PORT_
I'm a little confused. I've tried to convert password to the utf8 in nsPKCS12Blob::ExportToFile there [3]. This produces .p12 files which I'm unable to open (saying invalid password). Or did you mean to use utf8 encoded password just for the SEC_PKCS12AddCertAndKey and not for SEC_PKCS12CreatePasswordPrivSafe [4]?


[1] https://searchfox.org/mozilla-central/source/security/nss/lib/pkcs12/p12e.c#356
[2] https://searchfox.org/mozilla-central/source/security/nss/lib/pkcs12/p12local.c#1005
[3] https://searchfox.org/mozilla-central/source/security/manager/ssl/nsPKCS12Blob.cpp#116
[4] https://searchfox.org/mozilla-central/source/security/manager/ssl/nsPKCS12Blob.cpp#172

(In reply to Jan Horak from comment #15)
> > register a UTF8 to UCS2 with NSS. The NSS interface was originally an ASCII
> > to UCS2 encoder. If you do the full UTF8 to ucs2, then you'll be able to
> > maintain full fidelity with your UCS2 input. I imagine firefox already has
> > such a decoder internally. If not NSS supplies implementations in SEC_PORT_
> I'm a little confused. I've tried to convert password to the utf8 in
> nsPKCS12Blob::ExportToFile there [3]. This produces .p12 files which I'm
> Or did you mean to use utf8 > encoded password just for the SEC_PKCS12AddCertAndKey and not for
> SEC_PKCS12CreatePasswordPrivSafe ?
Hm, does not make a sense, both use sec_pkcs12_encode_password to convert password.

OK, so I think what is happening is Firefox is supplying the password in UCS2. NSS is expecting it in UTF8. In some cases, the actual pkcs12 encoding needs to be UCS2, so NSS calls a function which converts UTF8->UCS2. This function is replaceable. Firefox versions doesn't convert, but just copies. This fails in cases where we need UTF8 in the encoding.

I believe Firefox is passing UCS2 (raw) to NSS, and then supplying a NULL conversion. This works for the old PKCS 12, but not AES. It looks like you confirmed that firefox is, in fact, passing UCS2. Now we need to change the encoder that firefox supplies. There are two cases where the encoder could be called: 1) for the integrity hash, and 2) for the actual encryption. The integrity hash is encoded in UCS2 and the actual encryption is UTF8 for AES and UCS2 for 3DES. If you pass UTF8 in and have a UTF8-UCS2 encoder defined then NSS manages this all for you automatically (sec_pkcs12_encode_password() knows which to use based on the password function).

So the easiest thing to do is to 1) pass UTF8 into NSS, and 2) change PORT_SetUCS2_ASCIIConversionFunction() to take p12u_ucs2_ascii_conversion_function shown below (copied from pk12util. If firefox has an equivalent function, you can use it as well):

static SECStatus
p12u_SwapUnicodeBytes(SECItem *uniItem)
{
    unsigned int i;
    unsigned char a;
    if ((uniItem == NULL) || (uniItem->len % 2)) {
        return SECFailure;
    }
    for (i = 0; i < uniItem->len; i += 2) {
        a = uniItem->data[i];
        uniItem->data[i] = uniItem->data[i + 1];
        uniItem->data[i + 1] = a;
    }
    return SECSuccess;
}


static PRBool
p12u_ucs2_ascii_conversion_function(PRBool toUnicode,
                                    unsigned char *inBuf,
                                    unsigned int inBufLen,
                                    unsigned char *outBuf,
                                    unsigned int maxOutBufLen,
                                    unsigned int *outBufLen,
                                    PRBool swapBytes)
{
    SECItem it = { 0 };
    SECItem *dup = NULL;
    PRBool ret;
    it.data = inBuf;
    it.len = inBufLen;
    dup = SECITEM_DupItem(&it);
    if (!dup) {
        return PR_FALSE;
    }
    /* If converting Unicode to ASCII, swap bytes before conversion
     * as neccessary.
     */
    if (!toUnicode && swapBytes) {
        if (p12u_SwapUnicodeBytes(dup) != SECSuccess) {
            SECITEM_ZfreeItem(dup, PR_TRUE);
            return PR_FALSE;
        }
    }
    /* Perform the conversion. */
    ret = PORT_UCS2_UTF8Conversion(toUnicode, dup->data, dup->len,
                                   outBuf, maxOutBufLen, outBufLen);
    SECITEM_ZfreeItem(dup, PR_TRUE);
    return ret
}

I've just tried the suggested code and it seems to be working:
❯ openssl pkcs12 -passin pass:mojeheslo -in ~/b1.p12 -out /dev/null -info -noout
MAC: sha1, Iteration 600000
MAC length: 20, salt length: 16
PKCS7 Data
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 600000, PRF hmacWithSHA1
PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA1
Certificate bag
Certificate bag

Thanks a lot. Hubert is it what you're expecting to get?

yes, that's good enough

Upstream bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1694689

Created attachment 1762108 [details]
exported certificate which cannot be imported to firefox

I'm unable to import the newly exported certificate. The import fails with SEC_ERROR_BAD_DER (at least what PR_GetError() returns). What could be wrong?

Currently the patch has been accepted by upstream, with the Firefox 91 esr we'll be able to:

Set the 'security.pki.use_modern_crypto_with_pkcs12' preference to true if user wants to backup
own certificates to the file using AES-128-CBC (for the password) and AES_256_CBC (for the certificate) algorithms.

i took the export_new.p12, imported it to ff 91.2.0-4, backed it up to backup.p12 and verified with  openssl pkcs12 -passin pass:mojeheslo -in backup.p12 -out /dev/null -info -noout

here's the output:
MAC: sha1, Iteration 600000
MAC length: 20, salt length: 16
PKCS7 Data
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 600000, PRF hmacWithSHA1
PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF hmacWithSHA1
Certificate bag
Certificate bag

also the security.pki.use_modern_crypto_with_pkcs12 is set to true in about:config

if everybody agrees, i'll switch this bz to verified.

(In reply to Jiri Prajzner from comment #35)
> i took the export_new.p12, imported it to ff 91.2.0-4, backed it up to
> backup.p12 and verified with  openssl pkcs12 -passin pass:mojeheslo -in
> backup.p12 -out /dev/null -info -noout
> 
> here's the output:
> MAC: sha1, Iteration 600000
> MAC length: 20, salt length: 16
> PKCS7 Data
> Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 600000, PRF
> hmacWithSHA1
> PKCS7 Encrypted data: PBES2, PBKDF2, AES-128-CBC, Iteration 600000, PRF
> hmacWithSHA1
> Certificate bag
> Certificate bag
> 
> also the security.pki.use_modern_crypto_with_pkcs12 is set to true in
> about:config
> 
> if everybody agrees, i'll switch this bz to verified.

ack

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: firefox), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2311