Bug 2160797 - openssl smime and cms commands default to 3DES and PKCS#1v1.5 encryption in FIPS mode
Summary: openssl smime and cms commands default to 3DES and PKCS#1v1.5 encryption in F...
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: openssl
Version: 9.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: rc
: ---
Assignee: Dmitry Belyavskiy
QA Contact: Hubert Kario
Mirek Jahoda
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-13 18:14 UTC by Hubert Kario
Modified: 2023-07-25 18:37 UTC (History)
1 user (show)

Fixed In Version: openssl-3.0.7-18.el9
Doc Type: Known Issue
Doc Text:
Cause: OpenSSL cms and smime commands default to legacy algorithms, like 3DES for symmetric encryption and PKCS#1 v1.5 for asymmetric encryption. Consequence: Those algorithms are forbidden in FIPS mode, thus the operations fail. Workaround (if any): There is no workaround for use of PKCS#1 v1.5 encryption in smime command. For cms command, it's possible to specify the recipient using the -recip switch and then set the padding used using -keyopt rsa_padding_mode:oaep to use OAEP. For use of 3DES, both commands accept accept -aes128 and -aes256 switch to change the symmetric algorithm. Result: Encrypting files using cms and smime commands in FIPS mode doesn't work with default settings.
Clone Of:
Environment:
Last Closed:
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker CRYPTO-9281 0 None None None 2023-01-16 12:16:48 UTC
Red Hat Issue Tracker CRYPTO-9282 0 None None None 2023-06-20 13:17:06 UTC
Red Hat Issue Tracker RHELPLAN-145084 0 None None None 2023-01-13 18:16:42 UTC

Description Hubert Kario 2023-01-13 18:14:09 UTC 
Description of problem:
When the openssl smime command is used to encrypt email in FIPS mode, it still uses 3DES. Since that algorithm is not allowed in FIPS mode, the operation fails.

Version-Release number of selected component (if applicable):
openssl-3.0.7-2.el9.x86_64

How reproducible:
always

Steps to Reproduce:
Steps to Reproduce:
1. wget https://raw.githubusercontent.com/redhat-qe-security/certgen/master/certgen/lib.sh
2. source lib.sh
3. x509KeyGen ca
4. x509KeyGen client
5. x509SelfSign ca
6. x509CertSign --CA ca -t webclient client
7. echo 'Hello world' > message.txt
8. openssl smime -encrypt -in message.txt -out message.enc client/cert.pem

Actual results:
Error creating PKCS#7 structure
408C0F8F017F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:373:Global default library context, Algorithm (DES-EDE3-CBC : 27), Properties ()
408C0F8F017F0000:error:108C0100:PKCS7 routines:PKCS7_final:malloc failure:crypto/pkcs7/pk7_smime.c:80:

Expected results:
Email encrypted with an AES cipher and RSA-OAEP

Additional info:

setting -aes128 explicitly doesn't help as (I'm assuming) it's trying to use PKCS#1 v1.5 encryption with the provided RSA key, instead of RSA-OEAP:

Error creating PKCS#7 structure
40ACB4D62A7F0000:error:1C8000A5:Provider routines:rsa_encrypt:illegal or unsupported padding mode:providers/implementations/asymciphers/rsa_enc.c:163:
40ACB4D62A7F0000:error:108C0100:PKCS7 routines:PKCS7_final:malloc failure:crypto/pkcs7/pk7_smime.c:80:
Comment 1 Hubert Kario 2023-01-13 19:52:07 UTC 
Same issue for cms command:

# openssl cms -encrypt -in message.txt -out message.enc client/cert.pem

403C7406297F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:373:Global default library context, Algorithm (DES-EDE3-CBC : 27), Properties ()
403C7406297F0000:error:17000065:CMS routines:ossl_cms_EncryptedContent_init_bio:cipher initialisation error:crypto/cms/cms_enc.c:78:
403C7406297F0000:error:17000068:CMS routines:CMS_final:cms lib:crypto/cms/cms_smime.c:882:


# openssl cms -encrypt -in message.txt -out message.enc -aes128 client/cert.pem

404C8805B77F0000:error:1C8000A5:Provider routines:rsa_encrypt:illegal or unsupported padding mode:providers/implementations/asymciphers/rsa_enc.c:163:
404C8805B77F0000:error:17000074:CMS routines:cms_EnvelopedData_Encryption_init_bio:error setting recipientinfo:crypto/cms/cms_env.c:1142:
404C8805B77F0000:error:17000068:CMS routines:CMS_final:cms lib:crypto/cms/cms_smime.c:882:
Note You need to log in before you can comment on or make changes to this bug.