Bug 1971823

+++ This bug was initially created as a clone of Bug #1971533 +++

This kinda blocks bug #1938011.

When the FIPS mode is enabled and the non-glib code needs an MD5 checksum, the glib2 aborts in the gnutls code, because MD5 is disabled by the crypto-policies:

Thread 14 "pool" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff8b7fe700 (LWP 7298)]
gnutls_hmac (handle=0x0, ptext=0x7fff7c01ee70, ptext_len=26)
    at crypto-api.c:408
408		return _gnutls_mac((mac_hd_st *) handle, ptext, ptext_len);
#0  gnutls_hmac (handle=0x0, ptext=0x7fff7c01ee70, ptext_len=26)
    at crypto-api.c:408
#1  0x00007ffff5bbfde9 in g_hmac_update (hmac=hmac@entry=0x5555559a5d40, 
    data=data@entry=0x7fff7c01ee70 "J", length=length@entry=26)
    at ghmac-gnutls.c:128
#2  0x00007ffff5bbffbb in g_compute_hmac_for_data (
    digest_type=<optimized out>, key=<optimized out>, key_len=<optimized out>, 
    data=0x7fff7c01ee70 "J", length=26) at ghmac-utils.c:70
#3  0x00007ffff2257ff9 in calc_hmac_md5 () from /lib64/libsoup-2.4.so.1
#4  0x00007ffff2259355 in soup_auth_ntlm_get_connection_authorization ()
   from /lib64/libsoup-2.4.so.1
#5  0x00007ffff225cfe4 in update_authorization_header ()
   from /lib64/libsoup-2.4.so.1

The current upstream version of the glib2 does not use GnuTLS anymore, but the one in RHEL 8 does use it. It would be good to not use GnuTLS for MD5 checkum computation.

--- Additional comment from Michael Catanzaro on 2021-06-14 09:00:36 CDT ---

(In reply to Milan Crha from comment #0)
> The current upstream version of the glib2 does not use GnuTLS anymore, but
> the one in RHEL 8 does use it. It would be good to not use GnuTLS for MD5
> checkum computation.

Our downstream glib uses GnuTLS specifically because it's required for FIPS compliance. Upstream has never used GnuTLS. We have to either get our patch upstream somehow, or carry it downstream forever. See bug #1630260.

That said, GLib should not crash like this. It should crash with g_error() instead. I can refresh the patch to make that happen.

As for NTLMv2, let's discuss that in bug #1938011.

--- Additional comment from Michael Catanzaro on 2021-06-14 09:13:21 CDT ---

BTW GLib should be able to do MD5 checksums, via GChecksum with G_CHECKSUM_MD5, just fine regardless. Your code isn't just doing a checksum, it's doing HMAC, using GHmac. The HMAC is what triggers the RHEL crypto policy. So to be clear:

 * GChecksum: expected to succeed even if MD5 is disabled in system crypto policy. MD5 checksums should still work.
 * GHmac: expected to fail if MD5 is disabled in system crypto policy. Software that needs this is required to break.

At least, I think that's the desired behavior.

--- Additional comment from Milan Crha on 2021-06-14 09:15:27 CDT ---

Aha, it's a downstream patch. I thought it had been changed upstream long ago, because the RHEL 8.3 glib2 version is rather old. I didn't search for the custom/downstream patches at all.