Bug 2099651 - After updating nettle gnutls_certificate_allocate_credentials fails
Summary: After updating nettle gnutls_certificate_allocate_credentials fails
Alias: None
Product: Fedora
Classification: Fedora
Component: gnutls
Version: 36
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Red Hat Crypto Team
QA Contact: Fedora Extras Quality Assurance
: 2100400 (view as bug list)
Depends On:
TreeView+ depends on / blocked
Reported: 2022-06-21 12:23 UTC by Matej Marušák
Modified: 2022-07-07 06:53 UTC (History)
10 users (show)

Fixed In Version: gnutls-3.7.6-3.fc36
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2022-06-24 00:58:43 UTC
Type: Bug

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FC-484 0 None None None 2022-06-21 12:30:33 UTC

Description Matej Marušák 2022-06-21 12:23:11 UTC
Description of problem:

In our CI we have noticed that when we update nettle one of our libraries fails.

Version-Release number of selected component (if applicable):

Update nettle to 3.8-1.fc36 

How reproducible:

  gnutls_certificate_credentials_t creds = NULL;                                
  int r = gnutls_certificate_allocate_credentials (&creds);                     
  assert (r == GNUTLS_E_SUCCESS);

The assert fails.

Comment 1 Daiki Ueno 2022-06-21 12:37:01 UTC
Could you provide a complete reproducer? I cannot reproduce the issue with:

[root@164e77eefd44 ~]# rpm -qa nettle gnutls
[root@164e77eefd44 ~]# cat test.c
#include <gnutls/gnutls.h>
#include <assert.h>

int main (void) {
	gnutls_certificate_credentials_t creds = NULL;
	int r = gnutls_certificate_allocate_credentials(&creds);
	assert(r == GNUTLS_E_SUCCESS);
	return 0;
[root@164e77eefd44 ~]# gcc -o test test.c `pkg-config gnutls --cflags --libs`
[root@164e77eefd44 ~]# ./test ; echo $?

Comment 2 Martin Pitt 2022-06-21 12:58:23 UTC
Matej: This failure happens in our crypto-policies tests -- possibly you need to set a particular policy, or even enable FIPS?

Comment 3 Daiki Ueno 2022-06-21 13:53:38 UTC
> This failure happens in our crypto-policies tests -- possibly you need to set a particular policy, or even enable FIPS?

That sounds plausible; while gnutls_certificate_allocate_credentials merely calls out malloc/calloc functions, it can return error if the library is already in an unusable state, e.g., when FIPS self-tests are failed. In any case it would be helpful to collect debug logs with GNUTLS_DEBUG_LEVEL as described in:

Comment 4 Daiki Ueno 2022-06-21 14:03:44 UTC
Ah, I think I found the cause: gnutls recently switched the FIPS library integrity calculation to including all dependent libraries (including nettle), which is stored at: /usr/lib64/.gnutls.hmac:

After installing a newer nettle (or gmp), without updating the file there will be a mismatch.  I don't think we have a proper way to fix it other than rebuilding gnutls, but a workaround would be to set the GNUTLS_SKIP_FIPS_INTEGRITY_CHECKS envvar to 1.

Comment 5 Fedora Update System 2022-06-22 01:17:25 UTC
FEDORA-2022-b5e4fe1515 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-b5e4fe1515

Comment 6 Fedora Update System 2022-06-23 01:19:09 UTC
FEDORA-2022-b5e4fe1515 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-b5e4fe1515`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-b5e4fe1515

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Daiki Ueno 2022-06-23 10:31:38 UTC
*** Bug 2100400 has been marked as a duplicate of this bug. ***

Comment 8 Andreas Schneider 2022-06-23 10:50:37 UTC
Can't you create gating tests for nettle, gnutls etc. which just executes


Then you would detect issues earlier ...

Comment 10 Fedora Update System 2022-06-24 00:58:43 UTC
FEDORA-2022-b5e4fe1515 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 11 Matej Marušák 2022-07-07 06:53:58 UTC
Indeed it was in FIPS mode. Just removing mine needsinfo. Thank you for fixing it!

Note You need to log in before you can comment on or make changes to this bug.