------------------------ Context: ------------------------ System runs in FIPS mode: $ rpm -V libgcrypt openssl cupsd <no ouput> $ cat /proc/cmdline ro root=/dev/md2 elevator=deadline xencons=tty fips=1 $ cat /proc/sys/crypto/fips_enabled 1 $ ldd /usr/sbin/cupsd | grep -i -E 'gcry|ssl' libgcrypt.so.11 => /usr/lib64/libgcrypt.so.11 (0x00002b0c1249d000) libssl.so.6 => /lib64/libssl.so.6 (0x00002b0c12f50000) ------------------------ Description of problem: ------------------------ 1. -- When the system starts in FIPS mode - the cups daemon is complaining: $ tail -100 /var/log/messages | grep cups Sep 9 17:32:41 eve eggcups: Libgcrypt warning: custom allocation handler - FIPS mode inactivated Sep 9 17:40:17 eve cupsd: Libgcrypt warning: custom allocation handler - FIPS mode inactivated Sep 9 17:48:27 eve cupsd: Libgcrypt warning: custom allocation handler - FIPS mode inactivated the FIPS mode should stay activated! 2. -- And the daemon needs a lot of time to start $ time /etc/init.d/cups restart cups beenden: [ OK ] cups starten: [ OK ] real 1m21.976s user 0m0.012s sys 0m0.008s ------------------------ Version-Release number of selected component (if applicable): ------------------------ $ rpm -q libgcrypt openssl cups libgcrypt-1.4.4-5.el5 openssl-0.9.8e-20.el5 cups-1.3.7-26.el5_6.1 ------------------------ How reproducible: ------------------------ Boot in FIPS mode with cups daemon enabled ------------------------ Actual results: ------------------------ Start delay and warnings in /var/log/messages haven't more time to evaluate the implications of this behavior. ------------------------ Expected results: ------------------------ Cups daemon starts like in nonFIPS mode. ------------------------ Additional info: ------------------------ About FIPS mode: http://people.redhat.com/sgrubb/files/RHEL6-Security-Overview-2010.pdf Page 40 ------------------------ Thanks
This is due to gnutls calling gcry_set_allocation_handler().
Yes, gnutls could be changed by backporting from the more recent upstream that does not set the allocation handler in libgcrypt. However I am not sure the delay is directly related to the inactivation of the FIPS mode of libgcrypt. It might be that the delay would still be there even if the FIPS mode of libgcrypt would not be inactivated. There are integrity tests and algorithm selftests during the startup of the FIPS mode that might cause the delay.
The gcry_set_allocation_handler is not called anymore in the Red Hat Enterprise Linux 6 packages.
Thanks - what about RHEL 5?
We currently do not plan to fix this issue in Red Hat Enterprise Linux 5.