Description of problem: When using OAEP pkcs11 engine is sending incorrect CK_RSA_PKCS_OAEP_PARAM params: 0x00UL for the field "source" instead of 0x01UL (CKZ_DATA_SPECIFIED) See section 2.1.7 of PKCS #11 Cryptographic Token Interface Current Mechanisms Specification Version 3.0 https://docs.oasis-open.org/pkcs11/pkcs11-curr/v3.0/os/pkcs11-curr-v3.0-os.html#_Toc30061136 A strict HSM (Thales Luna) rejects this with CKR_MECHANISM_INVALID Version-Release number of selected component (if applicable): openssl-pkcs11-0.4.10-3.el8.x86_64 How reproducible: Always Steps to Reproduce: 1. Peform OAEP decryption openssl pkeyutl -decrypt -inkey 'pkcs11:token=mytoken;object=mykey' -keyform engine -engine pkcs11 -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in ciphertext -out recovered 2. 3. Actual results: CKR_MECHANISM_INVALID Expected results: Successful decryption Additional info: The params structure is 40 bytes long Incorrect: 60020000000000000300000000000000000000000000000000000000000000000000000000000000 Correct 60020000000000000300000000000000010000000000000000000000000000000000000000000000 The correct params are sent by Thales gem engine and python_pkcs11. The incorrect params are sent by openssl-pkcs11 Its possible (not verified) that softhsm2 - used in the development of openssl-pkcs11 - is more lenient in parsing this field so the "bug" is not detected
Verified that softhsm2 accepts C_DecryptInit calls with source=0 33: C_DecryptInit 2022-03-12 10:22:49.721 [in] hSession = 0x1 pMechanism->type=CKM_RSA_PKCS_OAEP pMechanism->pParameter->hashAlg=CKM_SHA_1 pMechanism->pParameter->mgf=CKG_MGF1_SHA1 pMechanism->pParameter->source=0 [out] pSourceData[ulSourceDalaLen] NULL [size : 0x0 (0)] [in] hKey = 0x2 Returned: 0 CKR_OK 34: C_Decrypt 2022-03-12 10:22:49.721 [in] hSession = 0x1 [in] pEncryptedData[ulEncryptedDataLen] 0000564542b5cc20 / 256 00000000 56 C5 1D 72 84 24 97 ED 87 B1 8B 26 C2 FF 3E 26 V..r.$.....&..>& 00000010 83 A1 DD 9E A4 3E EE 38 74 C1 C3 E4 B1 FE 1F 8B .....>.8t....... 00000020 07 1D AD 40 8D FB 31 65 F7 CA 1E 39 44 38 E0 C0 ...@..1e...9D8.. 00000030 40 57 49 F9 78 BD DA EB E0 51 EC F3 64 32 79 31 @WI.x....Q..d2y1
Thank you for the PR and a bug report. As we will have the issue merged upstream, I will look if and when we can update it in RHEL.
Also, please open a customer support case with Red Hat support to make sure the issue is properly prioritized. Without that, we can not guarantee the RHEL update.
Moving to RHEL 9 as we do not plan to do any other non-critical bugfixes in RHEL8.