Bug 737101

Summary:	start delay and warnings in cups while booting in FIPS mode
Product:	Red Hat Enterprise Linux 5	Reporter:	Leon Fauster <leonfauster>
Component:	gnutls	Assignee:	Tomas Mraz <tmraz>
Status:	CLOSED NEXTRELEASE	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	5.7	CC:	prc
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-03-05 14:46:13 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Leon Fauster 2011-09-09 16:00:20 UTC

------------------------
Context:
------------------------

System runs in FIPS mode:

$ rpm -V libgcrypt openssl cupsd
<no ouput>

$ cat /proc/cmdline 
ro root=/dev/md2 elevator=deadline xencons=tty fips=1

$ cat /proc/sys/crypto/fips_enabled
1

$ ldd /usr/sbin/cupsd | grep -i -E 'gcry|ssl'
	libgcrypt.so.11 => /usr/lib64/libgcrypt.so.11 (0x00002b0c1249d000)
	libssl.so.6 => /lib64/libssl.so.6 (0x00002b0c12f50000)


------------------------
Description of problem:
------------------------

1. --

When the system starts in FIPS mode - the cups daemon 
is complaining:

$ tail -100 /var/log/messages | grep cups
Sep  9 17:32:41 eve eggcups: Libgcrypt warning: custom allocation handler - FIPS mode inactivated
Sep  9 17:40:17 eve cupsd: Libgcrypt warning: custom allocation handler - FIPS mode inactivated
Sep  9 17:48:27 eve cupsd: Libgcrypt warning: custom allocation handler - FIPS mode inactivated

the FIPS mode should stay activated!


2. --

And the daemon needs a lot of time to start

$ time /etc/init.d/cups restart
cups beenden:                                           [  OK  ]
cups starten:                                             [  OK  ]

real	1m21.976s
user	0m0.012s
sys	0m0.008s




 
------------------------
Version-Release number of selected component (if applicable):
------------------------

$ rpm -q libgcrypt openssl cups
libgcrypt-1.4.4-5.el5
openssl-0.9.8e-20.el5
cups-1.3.7-26.el5_6.1




------------------------
How reproducible:
------------------------

Boot in FIPS mode with cups daemon enabled




------------------------  
Actual results:
------------------------

Start delay and warnings in /var/log/messages

haven't more time to evaluate the implications of this behavior. 



------------------------
Expected results:
------------------------

Cups daemon starts like in nonFIPS mode.



------------------------
Additional info:
------------------------

About FIPS mode:

http://people.redhat.com/sgrubb/files/RHEL6-Security-Overview-2010.pdf

Page 40



------------------------
Thanks

Comment 1 Tim Waugh 2011-09-15 15:55:34 UTC

This is due to gnutls calling gcry_set_allocation_handler().

Comment 2 Tomas Mraz 2011-09-15 18:59:14 UTC

Yes, gnutls could be changed by backporting from the more recent upstream that does not set the allocation handler in libgcrypt.
However I am not sure the delay is directly related to the inactivation of the FIPS mode of libgcrypt. It might be that the delay would still be there even if the FIPS mode of libgcrypt would not be inactivated. There are integrity tests and algorithm selftests during the startup of the FIPS mode that might cause the delay.

Comment 3 Tomas Mraz 2012-03-05 14:46:13 UTC

The gcry_set_allocation_handler is not called anymore in the Red Hat Enterprise Linux 6 packages.

Comment 4 Leon Fauster 2012-03-05 15:11:07 UTC

Thanks - what about RHEL 5?

Comment 5 Tomas Mraz 2012-03-05 15:19:22 UTC

We currently do not plan to fix this issue in Red Hat Enterprise Linux 5.