Bug 524794 - Wrong check for fips_enabled breaks login on kernels with CONFIG_CRYPTO_FIPS=n
Summary: Wrong check for fips_enabled breaks login on kernels with CONFIG_CRYPTO_FIPS=n
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: nss-softokn
Version: 12
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Elio Maldonado Batiz
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 504949 590199
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-09-22 08:33 UTC by Michal Schmidt
Modified: 2010-09-25 00:38 UTC (History)
4 users (show)

Fixed In Version: 3.12.4-19.fc12
Clone Of:
Environment:
Last Closed: 2010-09-25 00:38:38 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Michal Schmidt 2009-09-22 08:33:14 UTC
Description of problem:
I needed to test an upstream kernel, so I built one from current Linus's git. In its configuration I disabled several options to save time.
After successful installation and reboot the kernel booted fine. However I could login neither in gdm nor in a VT. It seemed like I had a wrong password, but I did not.
After some debugging I found that unix_chkpwd returned code 7 (PAM_AUTH_ERR) even when given a correct password.
Digging deeper, I found that NSSLOW_Init() was failing. Further investigation lead me to nsslow_GetFIPSEnabled() where I found this:

    f = fopen("/proc/sys/crypto/fips_enabled", "r");
    if (!f)
        return 1;

It means if the kernel is not FIPS aware, NSS goes to FIPS mode. This cannot be right. It would make sense to return 0, not 1.

(Among the options I had disabled in the kernel configuration was CONFIG_CRYPTO_FIPS whose help text says: "You should say no unless you know what this is." Since I have no idea what FIPS is, I said no to it.
As a workaround, I recompiled the kernel with CONFIG_CRYPTO_FIPS enabled. That made login work.)

Version-Release number of selected component (if applicable):
nss-3.12.4-9.fc12.x86_64
nss-softokn-3.12.4-9.fc12.x86_64
pam-1.1.0-5.fc12.x86_64

How reproducible:
always

Steps to Reproduce:
1. Build a kernel without CONFIG_CRYPTO_FIPS.
2. Install it, reboot.
3. Try to login with the correct password.
  
Actual results:
PAM authentication fails.

Expected results:
Authentication should succeed.

Comment 1 Bob Relyea 2009-09-22 17:38:39 UTC
Hmm, why does PAM fail, though?
Sure, libfreebl may be in FIPS mode, but that just means it's doing a bunch of checks, all of which should pass.

Elio, have you tried turning FIPS mode on in rawhide?


bob

Comment 2 Elio Maldonado Batiz 2009-09-22 21:54:15 UTC
In Rawhide I enabled FIPS mode by passing fips=1 at boot time and PAM authentication fails with the correct password. The problem did not occur in my F-11 system which uis also up to NSS 3.12.4.

Comment 3 Bob Relyea 2009-09-22 22:03:17 UTC
Thanks. This means we need to fix this fips=1 bug, which would make the main issue described here less of one.

The only issues that could cause this is 1) Some bug in the hash here it's failing the POST (very unlikely). or 2) something is wrong with the .chk file (it's corrupted, unavailable, etc.

bob

Comment 4 Michal Schmidt 2009-09-22 22:34:29 UTC
By tracing unix_chkpwd in gdb I can see that the .chk file is available, BLAPI_SHVerify() opens it, reads it, and calculates the SHA1 from it (I don't know if it gets the right result though), but DSA_VerifyDigest() returns SECFailure.

Comment 5 Michal Schmidt 2009-09-22 22:35:59 UTC
(In reply to comment #4)
> By tracing unix_chkpwd in gdb I can see that the .chk file is available,
> BLAPI_SHVerify() opens it, reads it, and calculates the SHA1 from it

Correction: the SHA1 is calculated from the .so library.

Comment 6 Bob Relyea 2009-09-22 22:50:13 UTC
yes, I just verified the .chk files (all of them) are not being generated correctly in the package.

You can verify this is the case by trying the following:

mkdir /tmp/nsstest
certutil -N -d /tmp/nsstest
{supply a random password --- twice)
modutil -fips true -dbdir /tmp/nsstest

The last command should turn on NSS fips mode, but it will fail.

You can work around this by running the following:
/usr/lib/nss/unsupported/shlibsign -v -i /usr/lib/libnssdbm3.so
/usr/lib/nss/unsupported/shlibsign -v -i /usr/lib/libfreebl3.so
/usr/lib/nss/unsupported/shlibsign -v -i /usr/lib/libsoftokn3.so

(on 64 bit platforms change /lib/ to /lib64/ in all cases above).

Elio will figure out what is wrong with the spec file.

bob

Comment 7 Michal Schmidt 2009-09-23 06:42:28 UTC
Prelink changes the hash results.

$ rpm -ql nss-softokn|grep '.so$'
/usr/lib64/libnssdbm3.so
/usr/lib64/libsoftokn3.so

$ cat /etc/prelink.conf.d/nss-softokn-prelink.conf 
-b /lib{,64}/libfreebl3.so
-b /lib{,64}/libsoftokn3.so
-b /lib{,64}/libnssdbm3.so

Notice the lack of "/usr" prefix in the config file.

Comment 8 Elio Maldonado Batiz 2009-09-23 15:04:22 UTC
Thank you Michal, that's likely the cause. I am now testing the fix based on a scratch build of nss-softokn at 
http://koji.fedoraproject.org/koji/taskinfo?taskID=1700822
I can now log in without problems when I set fips=1. 
A real build coming later once I test some more.

Comment 9 Elio Maldonado Batiz 2009-09-23 16:53:20 UTC
Th build at http://koji.fedoraproject.org/koji/taskinfo?taskID=1701188 has the fix. I'm updating this bug from a system with fips=1. I have noticed other problems but these are with OpenSSL.
 
[root@nss4you ~]# yum update
Loaded plugins: presto, refresh-packagekit
rawhide/metalink                                         | 8.5 kB     00:00     
fips.c(152): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE
Aborted

I'll check whether there is a bug on these and log if need be.

Comment 10 Tomas Mraz 2009-09-23 17:25:35 UTC
We do not have any prelink.conf files to exclude openssl from prelinking. As openssl is dependency of huge number of packages it would make the prelinking unuseful. This means you should always disable prelink and undo it (as done by the prelink cron job when prelink is disabled) if you want to use the FIPS mode.

Comment 11 Elio Maldonado Batiz 2009-09-30 00:01:24 UTC
Tomas,  This is the procedure I followed to disable prelink for whe in FIPS mode"
1. Edited/etc/sysconfig/prelink and changed the line that says PRELINKING=yes to be PRELINKING=no.
2. I ran prelink -au.
(By the way, I get "prelink: /usr/lib/debug/usr/bin/lemon.debug: PT_INTERP segment not corresponding to .interp section
prelink: /usr/lib/debug/usr/bin/sqlite3.debug: PT_INTERP segment not corresponding to .interp section". Don't know how serious this may be)
3. Edited /boot/grub/grub.conf. Increased the timeout and pre-appended a # to 
   hiddenmenu so I could break in at boot time.
4. Rebooted and I edited the kernel line by adding fips=1 and continued to boot
5. Once logged in I did a cat /proc/sys/crypto/fips-enabled and so a "1"
grep /etc/sysconfig/prelink shows me "PRELINKING=no"
6. Next I do 'su yum update' and that's when get 
fips.c(152): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST
FAILURE.
I reran "prelink -a" again and tried the yum update again but that didn't help.
Is there something else I should have done?

Comment 12 Tomas Mraz 2009-09-30 18:36:17 UTC
I found a problem in the FIPS selftest of RSA algorithm in openssl-1.0.0-0.7.beta3. I'm currently building a fixed package and will request moving it to F12.

Comment 13 Bug Zapper 2009-11-16 12:44:54 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle.
Changing version to '12'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 14 Elio Maldonado Batiz 2010-05-11 23:08:26 UTC
The following updates will fix this bug.

http://admin.fedoraproject.org/updates/nss-softokn-3.12.4-19.fc12
http://admin.fedoraproject.org/updates/prelink-0.4.3-2.fc12


Note You need to log in before you can comment on or make changes to this bug.