Bug 2086562

Summary: libotr should use gcry_pk_hash_sign()/gcry_pk_hash_verify() instead of gcry_pk_sign()/gcry_pk_verify()
Product: Red Hat Enterprise Linux 9 Reporter: Clemens Lang <cllang>
Component: libotrAssignee: Joe Orton <jorton>
Status: CLOSED WONTFIX QA Contact: František Hrdina <fhrdina>
Severity: low Docs Contact: Šárka Jana <sjanderk>
Priority: unspecified    
Version: 9.1CC: fhrdina, sjanderk, tkorbar
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
.`libotr` is not compliant with FIPS The `libotr` library and toolkit for off-the-record (OTR) messaging provides end-to-end encryption for instant messaging conversations. However, the `libotr` library does not conform to the Federal Information Processing Standards (FIPS) due to its use of the `gcry_pk_sign()` and `gcry_pk_verify()` functions. As a result, you cannot use the `libotr` library in FIPS mode.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2023-02-14 14:47:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Clemens Lang 2022-05-16 12:37:11 UTC 
Description of problem:
In libotr-4.1.1/src/privkey.c, libotr uses gcry_pk_sign() and gcry_pk_verify() from libgcrypt. This API is not FIPS-compliant, because hash and signature are computed separately and not together inside of the boundary of the libgcrypt FIPS module. You should consider using the gcry_pk_hash_sign() and gcry_pk_hash_verify() functions introduced in libgcrypt-1.10 instead.

Version-Release number of selected component (if applicable):
4.1.1-13.el9

How reproducible:
grep -rE 'gcry_pk_(sign|verify)' .

Actual results:
gcry_pk_sign() and gcry_pk_verify() are used.

Expected results:
gcry_pk_hash_sign() and gcry_pk_hash_verify() are used to compute the digest to be signed inside of the FIPS module boundary.

Additional info:
See https://dev.gnupg.org/T4894, where this function was introduced. From what I can see, it should be possible to use it with an HMAC, although it seems that was never discussed upstream.

I don't know if libotr is supported in FIPS mode.