Bug 2142121

Summary:	In FIPS mode, openssl should reject SHAKE as digest for RSA-OAEP or provide an indicator
Product:	Red Hat Enterprise Linux 9	Reporter:	Clemens Lang <cllang>
Component:	openssl	Assignee:	Clemens Lang <cllang>
Status:	CLOSED ERRATA	QA Contact:	Alicja Kario <hkario>
Severity:	medium	Docs Contact:	Jan Fiala <jafiala>
Priority:	high
Version:	9.0	CC:	cllang, gfialova, hkario, ssorce
Target Milestone:	rc	Keywords:	Triaged, ZStream
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	openssl-3.0.7-5.el9	Doc Type:	Deprecated Functionality
Doc Text:	.OpenSSL in FIPS mode cannot use SHAKE-128 and SHAKE-256 for composite operations The National Institute of Standards and Technology (NIST) allows only the SHAKE algorithms to be used as standalone functions. With this update, OpenSSL in FIPS mode can no longer use the SHAKE-128 and SHAKE-256 expandable output functions for composite operations, such as message digest signatures and mask generation functions.	Story Points:	---
Clone Of:
Clones:	2144010 2144011 (view as bug list)		Environment:
Last Closed:	2023-05-09 08:20:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2144010, 2144011

Description Clemens Lang 2022-11-11 17:16:31 UTC

Description of problem:
According to FIPS 140-3 IG, section C.C, while SHA-3 may be used in higher-level algorithms (such as RSA-OAEP and RSASSA-PSS), SHAKE128 and SHAKE256 may not:

"To be used in an approved mode of operation, the SHA-3 hash functions may be implemented either as part of an approved higher-level algorithm, for example, a digital signature algorithm, or as the standalone functions. The SHAKE128 and SHAKE256 extendable-output functions may only be used as the standalone algorithms."

Version-Release number of selected component (if applicable):
3.0.1-43.el9_0

How reproducible:
Attempt to use RSA-OAEP encryption with one of the SHAKE digests in rsa_oaep_md.

Steps to Reproduce:
1. openssl genpkey -algorithm rsa -pkeyopt rsa_keygen_bits:2048 -out rsa2048.pem
2. openssl rand -out secret 64
3. openssl pkeyutl -encrypt -inkey rsa2048.pem -in secret -out wrapped-secret -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:shake256

Actual results:
Encryption succeeds

Expected results:
pkeyutl: Can't set parameter "rsa_oaep_md:md5":
802BCA725F7F0000:error:0308010C:digital envelope routines:?????:digest not allowed

Additional info:
https://csrc.nist.gov/CSRC/media/Projects/cryptographic-module-validation-program/documents/fips%20140-3/FIPS%20140-3%20IG.pdf

Comment 17 errata-xmlrpc 2023-05-09 08:20:47 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: openssl security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2523