Bug 590362

+++ This bug was initially created as a clone of Bug #590199 +++

Description of problem:

After a "yum upgrade" today, I no longer have a working "su" or screensaver.
Neither accepts the password, even though I can log in using the same exact
password.

Version-Release number of selected component (if applicable):

I _think_ it's the nss-softokn-freebl-3.12.4-17.fc12.x86_64 upgrade.

But maybe I'm barking up the wrong tree.

How reproducible:

It's 100% for me.

Steps to Reproduce:
1. "su"
2. Type root password
3. 3s pause, followed by "Incorrect password"
  
Actual results:

I also get a very odd root-owned file called "-" in my home directory
when I do the above. I couldn't figure out where the heck it came from,
but with strace I saw 'su' ending up doing something like this:

   execve("/usr/sbin/prelink", "-u", "-o", "-", "/lib64/libfreebl3.so")

and that very much looks like it wants to prelink to stdout, but in fact
prelink will create a regular _file_ called "-", and copy that libfreebl3.so
data (along with owner/group/timestamp) into the new "-" file.

Which can't be right. So it looks to me like there's something really crazy
going on, and it does match the fact that nss-softokn-freebl got updated
today.


Expected results:

I expect to be root and install kernels or do yum upgrades to fix user
level bugs. 

Additional info:

--- Additional comment from torvalds on 2010-05-08 00:06:11 EDT ---

Btw, I also get some audit denied messages:

... avc:  denied  { getattr } for  pid=4287 comm="unix_chkpwd" path="/usr/sbin/prelink" ...

which sounds rather related.

--- Additional comment from torvalds on 2010-05-08 00:52:04 EDT ---

Hmm. This may have something to do with the fact that the kernel I am running does not have FIPS enabled. Just before the crazy prelink execve, it did

   open("/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)

and when that failed, it started looking at "lib64/libfreebl3.chk".

There's another bugzilla (#524794) about kernels without FIPS enabled having problems. That one was reported last September, but maybe the bug re-appeared?

I just tested, and with the default Fedora kernel, things work fine. So it does seem to have something to do with kernel version or configuration. Which would explain why not everybody is screaming.

--- Additional comment from torvalds on 2010-05-08 00:56:32 EDT ---

Confirmed. "su" starts working again if I enable CONFIG_CRYPTO_FIPS.

Anyway, please fix. Whatever tries to prelink libfreebl3 is some really horribly broken code, and something as fundamental as this shouldn't stop working (with no sane error messages) just because you don't have FIPS in the kernel.

--- Additional comment from felipe.contreras on 2010-05-08 09:37:57 EDT ---

Reverting back to nss-softokn{,-freebl} to 3.12.4-15 fixes the problem as well.

This was caused by the changes from bug #504949.

--- Additional comment from felipe.contreras on 2010-05-08 09:39:09 EDT ---

*** Bug 590269 has been marked as a duplicate of this bug. ***

--- Additional comment from felipe.contreras on 2010-05-08 11:47:16 EDT ---

Here's what's happening.

As mentioned in bug #524794, nss-softokn assumes FIPS is enabled when /proc/sys/crypto/fips_enabled doesn't exists, which happens when a kernel is built with CONFIG_CRYPTO_FIPS=n.

The fix is very simple:
     f = fopen("/proc/sys/crypto/fips_enabled", "r");
     if (!f)
-        return 1;
+        return 0;

I tested this, and now I can login.

Here's a koji build with the patch:
https://koji.fedoraproject.org/koji/taskinfo?taskID=2174243

However, in theory the same problems will happen when /proc/sys/crypto/fips_enabled is 1.

The problems can be explained by bug #504949 comment #43:
> I've changed prelink recently (starting with 0.4.3) to allow
> prelink -u -o - /lib/libfreebl3.so
> to output undo on stdout where the library when in fips strict mode can
> checksum it.

Therefore nss-softokn-3.12.4-17 should depend on prelink-0.4.3 which is not the case on F12, and the patch in nss-softokn-3.12.4-17 doesn't check for the prelink version.

--- Additional comment from torvalds on 2010-05-08 16:31:13 EDT ---

Can the fixed package please be pushed out asap? The current package is
clearly totally buggy, fedora-12 is not supposed to be some "rawhide" thing
that works or not based on random phases of the moon.

And why wasn't the "very simple" fix applied back in September of last year
when that bug was reported? Or was it applied and then lost again?

--- Additional comment from felipe.contreras on 2010-05-08 17:16:33 EDT ---

From what I can understand there are two problems:
1) CONFIG_CRYPTO_FIPS=n enables FIPS (has always been there)
2) enabling FIPS breaks the system with nss-softokn-3.12.4-17

F12 should work properly whether FIPS is enabled or not.

The problem reported on bug #524794 was *not* that CONFIG_CRYPTO_FIPS=n broke the system(1); it was the enabling FIPS broke the system. It seems that bug was solved when F12 was released, but things broke again with nss-softokn-3.12.4-17. So those are two separate bugs.

Apparently everyone forgot about bug #1.

We, the people that run a kernel with CONFIG_CRYPTO_FIPS=n, apparently are the only ones that noticed this problem (2). I couldn't care less about FIPS; I just want to be able to login to my system, pushing forward my proposed fix would achieve that.

--- Additional comment from torvalds on 2010-05-08 18:00:39 EDT ---

On Sat, 8 May 2010, Felipe Contreras  wrote:
> 
> The problem reported on bug #524794 was *not* that CONFIG_CRYPTO_FIPS=n broke
> the system(1); it was the enabling FIPS broke the system.

Incorrect.

524794 very clearly states that the bug was that 

        1. Build a kernel without CONFIG_CRYPTO_FIPS.
        2. Install it, reboot.
        3. Try to login with the correct password.

and that it was broken.

Now, 524794 then _also_ started talking about how enabling fips was broken 
even when fips was supported by the kernel, but the initial bug-report was 
all about CONFIG_CRYPTO_FIPS=n being broken.

And it clearly is _still_ broken, almost six months later. Whether it was 
ever fixed in the meantime is possible, of course.

--- Additional comment from felipe.contreras on 2010-05-08 18:17:14 EDT ---

(In reply to comment #9)
> On Sat, 8 May 2010, Felipe Contreras  wrote:
> > 
> > The problem reported on bug #524794 was *not* that CONFIG_CRYPTO_FIPS=n broke
> > the system(1); it was the enabling FIPS broke the system.
> 
> Incorrect.
> 
> 524794 very clearly states that the bug was that 
> 
>         1. Build a kernel without CONFIG_CRYPTO_FIPS.
>         2. Install it, reboot.
>         3. Try to login with the correct password.
> 
> and that it was broken.

Those were the steps to reproduce.

The bug was:
Wrong check for fips_enabled breaks login on kernels with CONFIG_CRYPTO_FIPS=n

Strictly speaking, that was fixed, since the wrong check for fips_enabled didn't break logins any more.

> Now, 524794 then _also_ started talking about how enabling fips was broken 
> even when fips was supported by the kernel, but the initial bug-report was 
> all about CONFIG_CRYPTO_FIPS=n being broken.

Since comment #1 the bug was used for fips=1 problems.

Clearly, on bug #524794 comment #3, Bob hijacked the efforts:
> This means we need to fix this fips=1 bug, which would make the main issue
> described here less of one.

Anyway, that's history, I hope we agree on two things:
1) CONFIG_CRYPTO_FIPS=n should not turn on FIPS
2) Whether or not FIPS works is irrelevant for this bug

Which bug is used to achieve 1), I don't care much, but in the package I created, I referenced this one.

--- Additional comment from emaldona on 2010-05-08 18:25:48 EDT ---

I realized I made a tyo in the spec file. See
https://bugzilla.redhat.com/show_bug.cgi?id=504949#c60

--- Additional comment from torvalds on 2010-05-08 18:29:34 EDT ---

> Those were the steps to reproduce.
>
> The bug was:
> Wrong check for fips_enabled breaks login on kernels with CONFIG_CRYPTO_FIPS=n

Having some serious reading problems, are you?

The text you quoted is EXACTLY about that CONFIG_CRYPTO_FIPS=n case.
And that is *the* case that is BROKEN right now.

So why do you claim that "it was fixed"? It clearly was not fixed. It's
the exact bug I hit yesterday.

What's your problem? You even quote that "CONFIG_CRYPTO_FIPS=n" being
clearly there, and then you go on to say over and over that it's about
"fips=1" even though IT CLEARLY WAS NOT.

Stop making excuses. Stop trying to make it about "fips=1", when it was
not at all about that. It was later _expanded_ to talk about that too,
but no, that clearly was _not_ the main bug report. And by clearly, I
say that both the main subject _and_ the first comment make it very very
clear that "CONFIG_CRYPTO_FIPS=n" was broken.

And it's still broken almost six months later. It's the thing my bug-report
also talks about. So please just make sure it gets fixed, and the fixed thing
gets pushed out. And don't try to make this about "fips=1", because it wasn't.
Not in the original bug report, and not in my bug report.

--- Additional comment from emaldona on 2010-05-08 19:00:41 EDT ---

A scratch build is at
http://koji.fedoraproject.org/koji/tasks?owner=emaldonado&state=closed&view=tree&method=all&order=-completion_time
My test:
[root@fedora12devel Downloads]# rpm -qa | grep ^nss-softokn
nss-softokn-3.12.4-10.fc12.x86_64
nss-softokn-freebl-3.12.4-17.fc12.x86_64
[root@fedora12devel Downloads]# rpm -qa | grep ^prelink
prelink-0.4.2-4.fc12.x86_64
When I try to update softokn:
Loaded plugins: auto-update-debuginfo, presto, refresh-packagekit
Setting up Local Package Process
Examining nss-softokn-3.12.4-18.fc12.x86_64.rpm: nss-softokn-3.12.4-18.fc12.x86_64
Marking nss-softokn-3.12.4-18.fc12.x86_64.rpm as an update to nss-softokn-3.12.4-10.fc12.x86_64
Found 35 installed debuginfo package(s)
Examining nss-softokn-freebl-3.12.4-18.fc12.x86_64.rpm: nss-softokn-freebl-3.12.4-18.fc12.x86_64
Marking nss-softokn-freebl-3.12.4-18.fc12.x86_64.rpm as an update to nss-softokn-freebl-3.12.4-17.fc12.x86_64
Resolving Dependencies
--> Running transaction check
---> Package nss-softokn.x86_64 0:3.12.4-18.fc12 set to be updated
---> Package nss-softokn-freebl.x86_64 0:3.12.4-18.fc12 set to be updated
--> Processing Conflict: nss-softokn-freebl-3.12.4-18.fc12.x86_64 conflicts prelink < 0.4.3
--> Finished Dependency Resolution
Error: nss-softokn-freebl conflicts with prelink
 ---- two more lines skipped ---
Which is what we want, nss-softokn won't be updated until prelink reaches 0.4.3.

--- Additional comment from torvalds on 2010-05-08 19:15:59 EDT ---

Why do you keep on harping on that "prelink" problem?

The first problem is that the whole prelink thing shouldn't have been done
AT ALL. 

The fact that "fips=1" was then _also_ broken is a totally separate bug.
I really don't care. I don't have FIPS enabled at all, I'm certainly not
going to care about "fips=1".

In other words - if "prelink" ends up being run just because I don't even
have FIPS enabled, it's a bug. It's a bug whether prelink is then fixed
or not. 

Please don't use this bugzilla entry to update prelink dependencies for the
package. I don't care. It's not why I opened the bug entry. Please make
sure that if the kernel doesn't have FIPS support, then nss-softokn-freebl
doesn't even _bother_ to do the check and the prelink reset. 

If you do that, and push out the fixed package, you can close the bugzilla
entry. I don't care one whit whether "fips=1" then works or not.

Ok?

--- Additional comment from felipe.contreras on 2010-05-08 19:32:44 EDT ---

(In reply to comment #12)
> > Those were the steps to reproduce.
> >
> > The bug was:
> > Wrong check for fips_enabled breaks login on kernels with CONFIG_CRYPTO_FIPS=n
> 
> Having some serious reading problems, are you?
> 
> The text you quoted is EXACTLY about that CONFIG_CRYPTO_FIPS=n case.
> And that is *the* case that is BROKEN right now.

No, it's about not being able to LOGIN _and_ CONFIG_CRYPTO_FIPS=n. The reporter himself identified the issue in bug #524794 #comment 3.

When nss-softokn-3.12.4-10 was released, the bug could be considered fixed because it met the expected results:

    Authentication should succeed.

If the bug had expected results:

    FIPS should be disabled when CONFIG_CRYPTO_FIPS=n

Then it would be clear that the bug wasn't fixed, but since that's not the case, it's not clear-cut.

But I don't care about that. The fact of the matter is that everyone forgot about the CONFIG_CRYPTO_FIPS=n part... that's all I was trying to explain.

> So why do you claim that "it was fixed"? It clearly was not fixed. It's
> the exact bug I hit yesterday.

There are 3 bugs. One of the bugs reported in bug #524794 was fixed, the other one is not. What you hit is a combination of two bugs, one that was introduced with nss-softokn-3.12.4-17.

> What's your problem? You even quote that "CONFIG_CRYPTO_FIPS=n" being
> clearly there, and then you go on to say over and over that it's about
> "fips=1" even though IT CLEARLY WAS NOT.

I'm saying that's how everyone threated it.

> Stop making excuses. Stop trying to make it about "fips=1", when it was
> not at all about that. It was later _expanded_ to talk about that too,
> but no, that clearly was _not_ the main bug report. And by clearly, I
> say that both the main subject _and_ the first comment make it very very
> clear that "CONFIG_CRYPTO_FIPS=n" was broken.
> 
> And it's still broken almost six months later. It's the thing my bug-report
> also talks about. So please just make sure it gets fixed, and the fixed thing
> gets pushed out. And don't try to make this about "fips=1", because it wasn't.
> Not in the original bug report, and not in my bug report.    

I'm not making excuses, I'm trying to explain what happened based on what I can see. Also, I have nothing to do with FIPS, or NSS, I'm just another user like you bitten by this bug. I read the two bugs, I think I understood the situation, and fixed it for me.

I have no interest in debating whether bug #524794 was really fixed, or partially fixed, or not fixed at all; I don't think it serves any purpose.

The only productive thing left to do IMO is to push the package with the fix, for which I've done all I could, now it's up to Fedora maintainers.

--- Additional comment from felipe.contreras on 2010-05-08 19:41:46 EDT ---

(In reply to comment #14)
> Please don't use this bugzilla entry to update prelink dependencies for the
> package. I don't care. It's not why I opened the bug entry. Please make
> sure that if the kernel doesn't have FIPS support, then nss-softokn-freebl
> doesn't even _bother_ to do the check and the prelink reset. 
> 
> If you do that, and push out the fixed package, you can close the bugzilla
> entry. I don't care one whit whether "fips=1" then works or not.
> 
> Ok?    

Elio is tackling this bug the way you reported it:
---
"su" and screensaver stopped working - not accepting passwords

expected results:
I expect to be root and install kernels or do yum upgrades to fix user
level bugs.
---

*Strictly* speaking, Elio would be fixing this bug if nss-softokn 3.12.4-18 depends on prelink >= 0.4.3, even if CONFIG_CRYPTO_FIPS=n turns FIPS on.

Which is clearly not what we want, so please change the summary of this bug to avoid confusion/hijacking.

--- Additional comment from felipe.contreras on 2010-05-08 19:46:32 EDT ---

@Elio how about picking my patch as well?
> Here's a koji build with the patch:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=2174243

Obviously CONFIG_CRYPTO_FIPS=n in the kernel should not turn FIPS on.

--- Additional comment from emaldona on 2010-05-08 20:49:58 EDT ---

(In reply to comment #17) It looks good to me, could you attach the patch to this bug?

--- Additional comment from felipe.contreras on 2010-05-08 20:55:41 EDT ---

Created an attachment (id=412590)
Fix for FIPS disabled in kernel

Proposed patch... should have been applied long time ago.

--- Additional comment from emaldona on 2010-05-08 21:13:13 EDT ---

(From update of attachment 412590 [details])
This patch lies inside the fips boundary.

--- Additional comment from emaldona on 2010-05-08 21:45:28 EDT ---

(From update of attachment 412590 [details])
The problem is that this patch makes  mssslow_GetFIPSEnabled return 1 (true the OS is FIPS enabled) when the file isn't there. That reverses the semantics of the call. The fix lies elsewhere.

--- Additional comment from emaldona on 2010-05-08 22:07:56 EDT ---

(In reply to comment #21) Oops, the only thing reversed are my eyes :-)
-        return 1;
+        return 0;
is indeed what we want to change.

--- Additional comment from updates on 2010-05-09 01:52:12 EDT ---

nss-softokn-3.12.4-18.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/nss-softokn-3.12.4-18.fc12

--- Additional comment from updates on 2010-05-09 01:54:56 EDT ---

nss-softokn-3.12.4-18.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/nss-softokn-3.12.4-18.fc13