Bug 2077799

Summary:	self-test fails in FIPS mode
Product:	Red Hat Enterprise Linux 9	Reporter:	Ondrej Moriš <omoris>
Component:	libkcapi	Assignee:	Zoltan Fridrich <zfridric>
Status:	NEW ---	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	9.0	CC:	omosnace
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ondrej Moriš 2022-04-22 09:38:02 UTC

Description of problem:

There are many failures in libkcapi self-test in FIPS mode. When FIPS mode is disabled the all tests pass. Self-test is actually testing kcapi and hence it might indicate problems in Kernel Crypto API itself.

Version-Release number of selected component (if applicable):

libkcapi-1.3.1-3.el9

How reproducible:

100% when FIPS mode is enabled.

Steps to Reproduce:

1. Install likcapi-tests (from the buildroot repository).

2. Disable 3DES cases 
   # sed -i 's/HASHEXEC\="1 2/HASHEXEC\="2/g' /usr/libexec/libkcapi/test.sh
   # sed -i 's/SYMEXEC\="1 2 3 4 5 6 7/SYMEXEC\="1 2 3/g' /usr/libexec/libkcapi/test.sh

3. Execute all tests:

   # /usr/libexec/libkcapi/test.sh 
   # /usr/libexec/libkcapi/kcapi-enc-test.sh 
   # /usr/libexec/libkcapi/kcapi-enc-test-large.sh
   # /usr/libexec/libkcapi/kcapi-dgst-test.sh
   # /usr/libexec/libkcapi/hasher-test.sh
   # /usr/libexec/libkcapi/kcapi-convenience.sh
   # /usr/libexec/libkcapi/kcapi-fuzz-test.sh

Actual results:

   # /usr/libexec/libkcapi/test.sh
   ...
   <fail>

   # /usr/libexec/libkcapi/kcapi-enc-test.sh
   ...
   Number of failures: 60

   # /usr/libexec/libkcapi/kcapi-enc-test-large.sh
   Number of failures: 0

   # /usr/libexec/libkcapi/kcapi-dgst-test.sh
   ...
   Number of failures: 64

   # /usr/libexec/libkcapi/hasher-test.sh
   ...
   Number of failures: 8

   # /usr/libexec/libkcapi/kcapi-convenience.sh
   ...
   Number of failures: 1

   # /usr/libexec/libkcapi/kcapi-fuzz-test.sh
   ...
   Number of failures: 0

Expected results:

All tests should pass.

Additional info:

The first test (test.sh) fails because is uses too short keys for SHA tests (for some reason RHEL-9 kernel needs longer keys in FIPS than in RHEL-8 FIPS or RHEL-9 non-FIPS):

# strace /usr/libexec/libkcapi/kcapi -m -x 8 -c 'hmac(sha1)' -k 73616c74 -p "70617373776f7264" -d 1 -b 20
...
setsockopt(3, SOL_ALG, ALG_SET_KEY, "password", 8) = -1 EINVAL (Invalid argument)
...

With longer password it will work (worked with 'RedHatEnterpriseLinux'). However, the other tests fails for different reasons I had no time to investigate.

Comment 1 Ondrej Moriš 2023-08-07 12:35:27 UTC

Update for kernel 5.14.0-347.el9 and the same version of libkcapi (1.3.1-3.el9) on RHEL-9.3 nightly compose - the only failing test in FIPS mode is now test.sh with the following failure:

libkcapi - Error: AF_ALG: bind failed (errno: -2)
libkcapi - Error: AF_ALG: bind failed (errno: -2)
libkcapi - Error: AF_ALG: bind failed (errno: -2)
[FAILED: 64-bit - 5.14.0-347.el9.x86_64] Auxiliary test failure detected

All the other tests are passing again.