Bug 852023

There are two things in play here: 
1) whether the system is on fips mode, queried via
/proc/sys/crypto/fips_enabled

Internally, freebl will query /proc/sys/crypto/fips_enabled and if if detects fips=1 it will preform the fips required self-integrity checks and if any fail will disable itself. 

2) whether the nss internal module itselfhas been cofigured to be on fips mode
The PK11_isFips() call answers this second question only.

From https://developer.mozilla.org/en-US/docs/NSS_reference/NSS_environment_variables
The evironment variable
NSS_FIPS String ("fips","true","on","1") Will start NSS in FIPS mode.

As far as I can tell the two things seem separate. The actions nss takes due to 1) don't seem to affect 2) as far as my reading of the code tells me. I could be wrong.

To my understanding, if you start the system in FIPS mode (fips=1 at kernel command line), you should not be able to use any non-FIPS mechanism. And the FIPS mode can be disabled only by rebooting the system. I see no point in FIPS mode if the libraries do not obey it.

Btw, I also tried setting NSS_FIPS=1 with no difference. It still works.

# NSS_FIPS=1 LDAPTLS_CIPHER_SUITE=EXP-RC2-CBC-MD5 ldapsearch -x -H ldaps://localhost

(EXP-RC2-CBC-MD5 is translated by libldap to SSL_RSA_EXPORT_WITH_RC2_CBC_40_MD5)

# NSS_FIPS=1 ./fipsmode 
FIPS mode is disabled.

Navigating the code some more I found that the nss-sysinit module also has a getFIPSMode function (same code as in freebl lowhash) at that has more consequences, see
http://mxr.mozilla.org/security/source/security/nss/lib/sysinit/nsssysinit.c#151

It's usage is in 218 get_list
http://mxr.mozilla.org/security/source/security/nss/lib/sysinit/nsssysinit.c#222

Notice that const char *nssflags = isFIPS ? nssDefaultFIPSFlags : nssDefaultFlags; is later used
i) set the user Database entry in the list of databses to open
  module_list[next++] = PR_smprintf(
    "library= "
    "module=\"NSS User database\" "
    "parameters=\"configdir='sql:%s' %s tokenDescription='NSS user database'\" "
    "NSS=\"trustOrder=75 %sflags=internal%s\"",
    userdb, stripped_parameters, nssflags,
    isFIPS ? ",FIPS" : "");

and 
ii) for the system database (opens db read-only unless the caller is root)
  const char *readonly = userCanModifySystemDB() ? "" : "flags=readonly";
  module_list[next++] = PR_smprintf(
   "library= "
   "module=\"NSS system database\" "
   "parameters=\"configdir='sql:%s' tokenDescription='NSS system database' %s\" "
      "NSS=\"trustOrder=80 %sflags=internal,critical\"",sysdb, readonly, nssflags);

How come there is no isFIPS ? ",FIPS" : "" part like we have for the userdb?

This get_list function is called by NSS_ReturnModuleSpecData(
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11load.c#424

but that happens after mod-isFIPS has been set at
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11load.c#379
     if (mod->isFIPS) {
         entry = (CK_C_GetFunctionList) 
                     PR_FindSymbol(softokenLib, "FC_GetFunctionList");
     } else {
         entry = (CK_C_GetFunctionList) 
                     PR_FindSymbol(softokenLib, "NSC_GetFunctionList");
     }

Does it matter? Having mod->isFIPS false means that we would be using 
NSC_GetFunctionList for entry instead instead of FC_GetFunctionList.
and the PK11_IsFIPS(void) function at
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11util.c#583
will return PR_FALSE even though the loaded internal module is the FIPS one
as the Jan found out.

Shouldn't we be resetting mod->isFIPS in case we discover that the system itself
is in FIPS mode?

Perhaps there is something I have missed in reading of the code. I haven't been able trace in the debugger yet as my attempts to install the latest RHEL-7 from nightly haven't been successful. Bob will set us straight.

> I believe that FIPS mode detection in NSS does not work. I tried OpenLDAP in 
> FIPS mode and I was able to use non-FIPS ciphers surprisingly. I'm attaching 
> simple reproducer to demonstrate.

1. Part of putting NSS into FIPS mode is disabling the usage of the non-FIPS ciphers. NSS FIPS mode is a mode that predates RHELs OS FIPS mode. As such it has it's own security policy. Part of that security policy says you are not supposed to use the non-FIPS ciphers, but the module doesn't enforce this. (This was done primarily because SSL used non-FIPS ciphers in a FIPS compliant way to do key derivation, turning those ciphers off would have broken SSL).

We can request an RFE to change that, though NSS upstream would have to determine that it won't break existing customers. Also note: such an RFE would, ironically, require a new FIPS validation.

I've been wanting to pull all the non-FIPS ciphers out of softoken and put them into their own token for a while now (that way we can update and add non-FIPS algorithms upstream without invalidating our FIPS validation).

> To my understanding, if you start the system in FIPS mode (fips=1 at kernel 
> command line), you should not be able to use any non-FIPS mechanism.

2. Putting the OS/system into FIPS mode doesn't necessarily put NSS into FIPS mode. NSS has it's own way of doing this (because NSS FIPS mode pre-dates the OS FIPS mode, and because the NSS FIPS mode has to work on other OSs that doen't have a system FIPS mode). If you are using the NSS system databases, then NSS sysinit is supposed to looks at the OS/System flag.

> Shouldn't we be resetting mod->isFIPS in case we discover that the
> system itself is in FIPS mode?

We could carry a Red Hat specific patch so that the system FIPS indicator partially overrides the NSS internal FIPS value. We'd have to do this carefully so that we don't break the NSS test suites which know how to put NSS into FIPS mode specifically. This change wouldn't require a reval, but also wouldn't really be pushable upstream (well maybe we could get it in on an ifdef, I wouldn't argue against that), since it's red hat specific. We'd have to be careful about how we do this. Blinding just following the kernel FIPS flag everywhere would require some careful thought. NSS's FIPS mode is a per-Slot affair (you can open both FIPS and non-FIPS slots in NSS). Also, many NSS applications (like Firefox) have their own FIPS UI's to put NSS into and out of FIPS mode. We need to make sure we don't confuse the user.

The other thing to worry about, is NSS requires you to authenticate to the token if you are in FIPS mode, because NSS is validated to level-2 (unlike the other modules). In order to authenticate, you must have already have a key database with a valid password set. If not, NSS doesn't let you do any crypto (so any application opening NSS up with NSS_NoDB() can't go into FIPS mode).

I think these issues are manageable (It's probably reasonable to assume that if FIPS mode is on system-wide, we must stay in FIPS mode. We can use some tricks to make Firefox disable it's 'Go into/out of FIPS mode' button, and disable the swith into/out of FIPS mode if the system-wide flag is set. We could fail NSS_NoDB() opens in that case as well. I highly suggest doing none of this in RHEL-6, of course.


Upshot: there are lots of good things that can be done here. There isn't any bandwidth to do them, but the NSS team is willing to review patches;).

bob

The way that the other libraries work is that when the fips enabled flag is seen, it puts the library into a soft fips mode. This is so that TLS continues to work. Applications that are not doing TLS should make an additional call to go into the strict compliance mode.

The only place where we deal with FIPS in OpenLDAP is when enabling the ciphers for TLS (tlsm_parse_ciphers):

http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=libraries/libldap/tls_m.c;h=1422ce26c0ab40c115edf91a7a948a36eae47168;hb=HEAD#l663

 686                       if (enabled == SSL_ALLOWED) {
 687                               if (PK11_IsFIPS() && !suite.isFIPS)    
 688                                       enabled = SSL_NOT_ALLOWED;
 689                       }

Is this approach incorrect then? How should we do that?

Version-Release number of selected component (if applicable):
nss-3.13.6-1.el7.x86_64


Steps to Reproduce:
1. boot to FIPS
2. initialize NSS
3. call PK11_CreateDigestContext(SEC_OID_MD5)
  

Actual results:
PK11_CreateDigestContext(SEC_OID_MD5) succeeds.


Expected results:
PK11_CreateDigestContext(SEC_OID_MD5) fails.


Additional info:
OpenSSL does not allow to initialize MD5 digest context when the system is running in FIPS.  NSS does not seem to care about MD5.  Does it mean that all users of NSS are supposed to check /proc/sys/crypto/fips_enabled and refuse to use MD5 on their own?

I'll defer to Bob Relya on FIPS matters for a most accurate answer.

Bob, do you have any update on this?

Sigh, I did, I guess the stupid bugzilla 'do you really want to post this' thing ate it.

The upshot... NSS is working as expected. NSS does not restrict non-FIPS ciphers mechanically, only by policy. NSS also does not trigger on /proc/sys/cyrpto/fips_enabled. The security policy for RHEL-5 is here: http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140sp/140sp815.pdf RHEL-6 is just finalizing, so I'll post it when it comes out, but it will be similiar.

NOTE that the NSS validation predates the /proc/sys/crypto/fips_enabled flags stuff: http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140val-all.htm#815

bob

My question was about RHEL-7.  So if I understand it correctly, NSS-based applications can use its MD5 implementation even if the system is running in FIPS mode and there is nothing to be fixed, right?

The application will not be FIPS compliant if it uses non-approved crypto algorithms.

The NSS FIPS mode status should be taken from NSS instead of the /proc/sys/crypto/fips_enabled. Bob could tell you how to do that.

And actually that is what OpenLDAP is doing - it's the PK11_IsFIPS() call.

You should be able to rely on either /proc/sys/crypto/fips_enabled or the PK11_IsFIPS call to tell wther NSS is in FIPS mode. That call with its PK11_SetFIPS(PRBool fipsMode) counterpart to set/unset the internal crypto module module fips mode by user request works on all platforms and preecedes the system wide query. On RHEL the cryptographic module when it is loading queries /proc/sys/crypto/fips_enabled and sets a flag to place itself on FIPS mode. 

This question comes to mind: Does the /proc/sys/crypto/fips_enabled setting override the old method? I would hope so, but I better check :-)

(In reply to comment #13)
> The application will not be FIPS compliant if it uses non-approved crypto
> algorithms.

At the same time, embedding the current FIPS policy in every application is a bad way to scale things.  If two years from now SHA-1 is prohibited, we will have to update all of the applications again.

Having the policy centralized in libraries, and letting applications just handle errors reported by the library, seems far more future-proof.

Sure, however that would require changes in NSS to reject the algorithms if the application tries to initialize them.

(In reply to comment #17)
> Sure, however that would require changes in NSS to reject the algorithms if
> the application tries to initialize them.

What was the reason to implement it such in OpenSSL but not in NSS?

There are applications out there supporting both the libraries and using a similar approach for both of them would decrease the maintenance overhead.

The way that libgcrypt does it is that there are 2 FIPS modes. A soft mode that allows MD5 for TLS purposes and a hard mode that does not. When it sees the fips_enabled flag, it goes into the soft mode. Because it does not know the intention of the application. The application can then decide that its not doing TLS and therefore ask for the more strict enforcement. This is how the aide program works...it asks for strict enforcement because it knows that its not doing TLS.

I prefer this kind of model where the app can say I want a strict technical barrier.

(In reply to comment #19)
> The way that libgcrypt does it is that there are 2 FIPS modes. A soft mode
> that allows MD5 for TLS purposes and a hard mode that does not. When it sees
> the fips_enabled flag, it goes into the soft mode. Because it does not know
> the intention of the application. The application can then decide that its
> not doing TLS and therefore ask for the more strict enforcement. This is how
> the aide program works...it asks for strict enforcement because it knows
> that its not doing TLS.
> 
> I prefer this kind of model where the app can say I want a strict technical
> barrier.

This removes the policy dependency from the applications, but only for applications that use TLS; those that do use TLS are left unprotected.  Even worse, a library now cannot rely on strict enforcement because it doesn't know whether any other part of the program uses TLS.

In my ideal world, the FIPS mode restrictions would just be universally enforced.

In practice we probably can't avoid making prohibited hash algorithms available in FIPS mode[1]; in that case the OpenSSL approach of making the application opt-in _for the specific digest operation_ (not process-wide) sounds like the most correct solution.


[1] Based on code reviews of some MD5 users, vast majority of of the "non-cryptographic uses" would be vulnerable to a successful attack on the hash as well.

libcurl uses both TLS and MD5 provided by NSS, so the switch to strict mode is not applicable anyway.  Based on the clarification I got from sec-standards-team, I will disable the NTML authentication explicitly based on the flag returned by PK11_IsFIPS(), which is independent of the kernel FIPS mode flag.  Note this will allow libcurl to use its internal MD4 implementation unless NSS is running in FIPS mode.

I agree that the approach taken by OpenSSL is most correct and useful.

openswan could go into the stricter NSS mode as it needs to exemption from TLS (through technically, there could be CRL/OCSP and pam auth requests that need TLS, those are handled by libcurl.

There is some logic build into openswan via Pluto_IsFIPS(), which does use the file /proc/sys/crypto/fips_enabled directly. It does not allow IPsec PSK based connections and it requires a password to unlock/protect RSA keys. I guess it would be difficult to build this logic into NSS, though perhaps we could put policy as part of a function in fipscheck-lib, so applications have a generic way of asking, and we can maintain fips policy in one location, the fips library.

I don't see anything about not using MD5, 1DES, or NULL encryption/hashing, or the smaller modp groups, although openswan per default already does not allow 1DES and modp-768. (And I do know there were issues with md5 being "required" for some things in fips mode that we are working on phasing out, so it must be limiting md5 somehow?

How can I switch NSS to FIPS mode so that PK11_IsFIPS() returns true?

I exported NSS_FIPS=1 to no avail:

https://developer.mozilla.org/en-US/docs/NSS_reference/NSS_environment_variables

The comment above the function does not explain me much:

 614│ /* Determine if we have the FIP's module loaded as the default
 615│  * module to trigger other bogus FIPS requirements in PKCS #12 and
 616│  * SSL
 617│  */
 618│ PRBool
 619│ PK11_IsFIPS(void)
 620│ {
 621│     SECMODModule *mod = SECMOD_GetInternalModule();
 622│
 623│     if (mod && mod->internal) {
 624├>        return mod->isFIPS;
 625│     }
 626│
 627│     return PR_FALSE;
 628│ }

Do we have any documentation of PK11_IsFIPS()?

We don't. Looking at the old nss api reference page at
http://www.mozilla.org/projects/security/pki/nss/ref/nssfunctions.html#utils
it just has a link to the implementation
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11util.c#588

A more informative page is what the nss-sysinit module does
http://mxr.mozilla.org/security/source/security/nss/lib/sysinit/nsssysinit.c#134
Another place is in NSSLowHash, part of nss-softoken-freebl, which was added for use by the glibc own hashing functions.

I wish PK11_IsFIPS() had a counterpart set function, something like a PK11_SetFIPS(PRBool fipsMode) function. That would only work with NSS's own internal module are there is no reliable way to set it on other modules external to NSS. PK11_InternalModuleSetFIPS would be a more accurate name.

The secmod tool sets the FIPS mode on and off but by by directly calling into softoken via the pk11 #11 layer to unload and reload the internal module. See
http://mxr.mozilla.org/security/source/security/nss/cmd/modutil/pk11.c
The same technique is used by JSS.

NSS determines fips mode by a setting in the database. FIPS mode is an attribute of softoken, not of NSS itself doesn't do crypto, and always operates in ways that tolerate FIPS mode modules.

> I wish PK11_IsFIPS() had a counterpart set function, something like a
 > PK11_SetFIPS(PRBool fipsMode) function. 

There is, sort of, You can swith to fips mode by deleting the internal module (it's an old historical hack that still exists)...

if (!PK11_IsFIPS()) {
   SECMODModule *mod = SECMOD_GetInternalModule();
   rv = SECMOD_DeleteInternalModule(mod->commonName);
   if (rv != SECSuccess) {
       /* do you're can't get to FIPS mode recovery here */
   }
}


> That would only work with NSS's own internal module are there is no
> reliable way to set it on other modules external to NSS.

FIPS only applies to the internal module. That's the only place NSS does crypto. Other modules have their own FIPS status if they do crypto. 

bob

Does not it enable FIPS for the running process only?

Is there a way to enable NSS' FIPS mode independently of the application?

It is not possible to legally switch into FIPS mode in an application. This is because there are "power on self tests" that must be performed before the crypto interface is available to the user of it. This is normally done in library's constructor function.

But what can be available is a function to say: "I'm not using weak crypto, so technically enforce restrictions on weak crypto (like MD5)." You can also drop out of FIPS mode - legally - and libraries can provide that as well. But you can only go into FIPS mode at program launch when it does the self tests.

Clear questions help get clear answers.

The normal way NSS triggers FIPS mode is based on the secomd.db/pkcs11.txt database. When you open NSS, you usually point to a database.

If you point to sql:/etc/pki/nssdb, then installing nsssysinit and putting the kernel in FIPS mode would do the trick.

If you open any other database, or you don't have nsssysinit installed, you can use modutil to tell the database to run in FIPS mode:

modutil -dbdir {your database} -fips on

If you don't open any database, then you as an application have to manage FIPS mode yourself as per comment 26

bob

(In reply to comment #30)
> If you point to sql:/etc/pki/nssdb, then installing nsssysinit and putting
> the kernel in FIPS mode would do the trick.

This does not work.  I have kernel FIPS mode enabled, nsssysinit enabled, but still getting 0 from PK11_IsFIPS().

> If you open any other database, or you don't have nsssysinit installed, you
> can use modutil to tell the database to run in FIPS mode:
> 
> modutil -dbdir {your database} -fips on

This works, but only if the sql:/ prefix is NOT used.  libcurl always uses the sql:/ prefix, so turning on FIPS is still impossible.

Created attachment 662399 [details]
test FIPS mode switch in NSS

# sh nss-fips.c
++ pkg-config nss --cflags --libs
+ gcc nss-fips.c -I/usr/include/nss3 -I/usr/include/nspr4 -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl
+ setup-nsssysinit.sh status
NSS sysinit is enabled
+ KFIPS=/proc/sys/crypto/fips_enabled
+ printf '%s = %s\n' /proc/sys/crypto/fips_enabled 1
/proc/sys/crypto/fips_enabled = 1
+ ./a.out sql:/etc/pki/nssdb
NSS_Init("sql:/etc/pki/nssdb") = 0
PK11_IsFIPS() = 0

++ mktemp
+ PASSWD=/tmp/tmp.0Pyn9Ci3WK
++ mktemp -d
+ DB=/tmp/tmp.RU71L7KBuQ
+ certutil -N -d /tmp/tmp.RU71L7KBuQ -f /tmp/tmp.0Pyn9Ci3WK
password file contains no data
+ echo
+ modutil -dbdir /tmp/tmp.RU71L7KBuQ -fips true
+ ./a.out /tmp/tmp.RU71L7KBuQ
NSS_Init("/tmp/tmp.RU71L7KBuQ") = 0
PK11_IsFIPS() = 1

+ ./a.out sql://tmp/tmp.RU71L7KBuQ
NSS_Init("sql://tmp/tmp.RU71L7KBuQ") = 0
PK11_IsFIPS() = 0

+ exit 0

Created attachment 662578 [details]
Enable FIPS mode if the system FIPS mode flag is set.

This patch forces NSS into FIPS mode if system fips mode bit is set.
 - If that bit is set, applications trying to switch out of FIPS mode will get and error code.
 - Applications that check to see if they can change modes (Like Firefox and Thunderbird) will be told it can't, so the firefox <Disable FIPS> button should be grayed out if the sytem fips mode bit is set.

If the bit is not set, NSS get's it's FIPS indication it's traditional way, so the Firefox 'Enable FIPS' button will be on as normal.

This but does not change NSS behavior WRT non-FIPS algorithms.

Still doesn't work in the latest compose.

Comment on attachment 662578 [details]
Enable FIPS mode if the system FIPS mode flag is set.

Output of the above checker with this patch applied:

$ sh -x nss-fips.c
+ set -x
++ pkg-config nss --cflags --libs
+ gcc nss-fips.c -I/usr/include/nss3 -I/usr/include/nspr4 -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl
+ setup-nsssysinit.sh status
NSS sysinit is enabled
+ KFIPS=/proc/sys/crypto/fips_enabled
+ printf '%s = %s\n' /proc/sys/crypto/fips_enabled 1
/proc/sys/crypto/fips_enabled = 1
+ ./a.out sql:/etc/pki/nssdb
NSS_Init("sql:/etc/pki/nssdb") = -1
PK11_IsFIPS() = 1

++ mktemp
+ PASSWD=/tmp/tmp.Xdhg1U8blt
++ mktemp -d
+ DB=/tmp/tmp.NA5TNfn9vD
+ certutil -N -d /tmp/tmp.NA5TNfn9vD -f /tmp/tmp.Xdhg1U8blt
password file contains no data
+ echo
+ modutil -dbdir /tmp/tmp.NA5TNfn9vD -fips true
FIPS mode already enabled.
+ ./a.out /tmp/tmp.NA5TNfn9vD
NSS_Init("/tmp/tmp.NA5TNfn9vD") = 0
PK11_IsFIPS() = 1

+ ./a.out sql://tmp/tmp.NA5TNfn9vD
NSS_Init("sql://tmp/tmp.NA5TNfn9vD") = 0
PK11_IsFIPS() = 1

+ exit 0

(In reply to Kamil Dudka from comment #44)
> Comment on attachment 662578 [details]
> Enable FIPS mode if the system FIPS mode flag is set.
> 
> Output of the above checker with this patch applied:
> 
> $ sh -x nss-fips.c
> + set -x
> ++ pkg-config nss --cflags --libs
> + gcc nss-fips.c -I/usr/include/nss3 -I/usr/include/nspr4 -lssl3 -lsmime3
> -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl
> + setup-nsssysinit.sh status
> NSS sysinit is enabled
> + KFIPS=/proc/sys/crypto/fips_enabled
> + printf '%s = %s\n' /proc/sys/crypto/fips_enabled 1
> /proc/sys/crypto/fips_enabled = 1
> + ./a.out sql:/etc/pki/nssdb
> NSS_Init("sql:/etc/pki/nssdb") = -1

Is this ^^^ failure of NSS_Init() expected/intended?

> Is this ^^^ failure of NSS_Init() expected/intended?

Only if one of the FIPS tests failed. Usually that's the integrity checks.

bob

(In reply to Bob Relyea from comment #47)
> > Is this ^^^ failure of NSS_Init() expected/intended?
> 
> Only if one of the FIPS tests failed. Usually that's the integrity checks.
> 
> bob

So why do the tests fail when initializing with "sql:/etc/pki/nssdb", but not with "/tmp/tmp.NA5TNfn9vD" then?  Can I trigger the failure without using your patch?

> So why do the tests fail when initializing with "sql:/etc/pki/nssdb", but not with "/tmp/tmp.NA5TNfn9vD" then? 

Hmm sounds like some funny sysinit problem. (Assuming that you have databased in /etc/pki/nssdb).

> Can I trigger the failure without using your patch?

It really depends on what is failing.

If you have sysinit turned on, it should have started things in FIPS mode if you used sql:/etc/pki/nssdb. I'm wondering if there isn't some funny interaction going on between the patch and sysinit. The big thing about sysinit is that it opens multiple NSS databases, not just the main database. That shouldn't cause an issue (It should be OK for the multiple databased to be openned in FIPS mode, but perhaps there is some sort of issue here.

I guess I should turn the question around, can you trigger the failure without the patch?

bob

Hi, any update on this? I see that the patch is already proposed and there are no objections as far as I can see (except c#45). Would it be possible to create a build for RHEL-7.0?

I don't see any reason not to make a build, but I don't think we can close this issue out until we understand why Kamil is having his failure.

We actually know why Kamil had the failure and I fixed that upstream. I was able to verify the fix on f18 quite some time ago. For rhel-7 we have to a wait ontil the packages get signed. Yes, we have to wait until it can be properly verified on RHEL-7.

(In reply to Elio Maldonado Batiz from comment #52)
> We actually know why Kamil had the failure and I fixed that upstream. I was
> able to verify the fix on f18 quite some time ago. For rhel-7 we have to a
> wait ontil the packages get signed. Yes, we have to wait until it can be
> properly verified on RHEL-7.

Ignore this comment, I confused this bug with another one that ist pending verification.(In reply to Bob Relyea from comment #51)
> I don't see any reason not to make a build, but I don'

> this issue out until we understand why Kamil is having his failure.
Agreed, we must understand this well. Also, I will make a scratch build with the proposed patch.

The failure does not occur without the patch.  It comes from here:

#0  secmod_ModuleInit (mod=0x611b30, reload=0x7fffffffdf98, alreadyLoaded=0x7fffffffde84) at pk11load.c:273
#1  0x00007ffff75fee8a in secmod_LoadPKCS11Module (mod=0x611b30, oldModule=0x7fffffffdf98) at pk11load.c:457
#2  0x00007ffff760d6e4 in SECMOD_LoadModule (modulespec=0x6081d0 "library= module=\"NSS system database\" parameters=\"configdir='sql:/etc/pki/nssdb' tokenDe
scription='NSS system database' flags=readonly\" NSS=\"trustOrder=80 cipherOrder=100 slotParams={0x00000001=[slotF"..., parent=0x607590, recurse=1) at pk11pa
rs.c:1014
#3  0x00007ffff760d7cf in SECMOD_LoadModule (modulespec=0x607380 "library=\"libnsssysinit.so\" name=\"NSS Internal PKCS #11 Module\" NSS=\"Flags=internal,mod
uleDBOnly,critical trustOrder=75 cipherOrder=100 slotParams=(1={slotFlags=[RSA,DSA,DH,RC2,RC4,DES,RANDOM,SHA1,MD5,"..., parent=0x606050, recurse=1) at pk11pa
rs.c:1049
#4  0x00007ffff760d7cf in SECMOD_LoadModule (modulespec=0x604cf0 "name=\"NSS Internal Module\" parameters=\"configdir='sql:/etc/pki/nssdb' certPrefix='' keyP
refix='' secmod='secmod.db' flags=readOnly,optimizeSpace updatedir='' updateCertPrefix='' updateKeyPrefix='' upd"..., parent=0x0, recurse=1) at pk11pars.c:10
49
#5  0x00007ffff75cc5cd in nss_InitModules (configdir=0x7fffffffe6f7 "sql:/etc/pki/nssdb", certPrefix=0x7ffff7702120 "", keyPrefix=0x7ffff7702120 "", secmodNa
me=0x7ffff7702385 "secmod.db", updateDir=0x7ffff7702120 "", updCertPrefix=0x7ffff7702120 "", updKeyPrefix=0x7ffff7702120 "", updateID=0x7ffff7702120 "", upda
teName=0x7ffff7702120 "", configName=0x0, configStrings=0x0, pwRequired=0, readOnly=1, noCertDB=0, noModDB=0, forceOpen=0, optimizeSpace=1, isContextInit=0)
at nssinit.c:435
#6  0x00007ffff75cc945 in nss_Init (configdir=0x7fffffffe6f7 "sql:/etc/pki/nssdb", certPrefix=0x7ffff7702120 "", keyPrefix=0x7ffff7702120 "", secmodName=0x7f
fff7702385 "secmod.db", updateDir=0x7ffff7702120 "", updCertPrefix=0x7ffff7702120 "", updKeyPrefix=0x7ffff7702120 "", updateID=0x7ffff7702120 "", updateName=
0x7ffff7702120 "", initContextPtr=0x0, initParams=0x0, readOnly=1, noCertDB=0, noModDB=0, forceOpen=0, noRootInit=0, optimizeSpace=1, noSingleThreadedModules
=0, allowAlreadyInitializedModules=0, dontFinalizeModules=0) at nssinit.c:639
#7  0x00007ffff75cccc5 in NSS_Init (configdir=0x7fffffffe6f7 "sql:/etc/pki/nssdb") at nssinit.c:748
#8  0x00000000004008ac in main (argc=2, argv=0x7fffffffe458) at nss-fips.c:34

Comment on attachment 662578 [details]
Enable FIPS mode if the system FIPS mode flag is set.

This patch breaks SSL in FIPS:

$ NSS_ALLOW_WEAK_SIGNATURE_ALG=1 curl -v https://bugzilla.redhat.com
* About to connect() to bugzilla.redhat.com port 443 (#0)
*   Trying 10.4.127.4...
* Connected to bugzilla.redhat.com (10.4.127.4) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* Server certificate:
*       subject: CN=*.redhat.com,OU=Web Operations,O=Red Hat Inc,L=Raleigh,ST=North Carolina,C=US,serialNumber=XeSJbTqpNUErLA/DI3Cs5qXyDcU1HYm2
*       start date: Nov 15 13:44:10 2011 GMT
*       expire date: Nov 17 11:36:58 2013 GMT
*       common name: *.redhat.com
*       issuer: CN=GeoTrust SSL CA,O="GeoTrust, Inc.",C=US
* NSS error -8182 (SEC_ERROR_BAD_SIGNATURE)
* Peer's certificate has an invalid signature.
* Closing connection 0
curl: (60) Peer's certificate has an invalid signature.

Upstream NSS as of 3.14 onward does not support MD5 for digital signatures. For RHEL-6 we have patch to restore accepting MD5 for them as we can't break compatibility on stable branches. For RHEL-7 we will reject MD5 for digital certificates. As is this a major RHEL update we don't have such constraints and security takes precedence. If I have understood Bob correctly, such is the intent, but let's ask Bob to be sure.

How does it relate to the patch we discuss here?  The above works fine with the NSS packages we ship in RHEL-7.  It breaks only if Bob's patch is applied, which I suspect was not intended.  Was it?

Elio, as Kamil points out, this bug is about automatically going into FIPS mode when this patch is included. Please set up a RHEL-7 system with an NSS built with patch and try to reproduce Kamil's problem.


Kamil: The error you are running into doesn't look like a failure to open the database, it looks like a failure to authenticate to the token. NSS fips is level-2, which means you must authenticate to the token to do any cryptographic operations (other than rng or hashing).

Try this without the patch:
modutil -fips true sql:/etc/pki/nssdb

Then retry your command. If it fails with the same message we are running into a curl mismatch in NSS FIPS mode.

bob

When saw that NSS_ALLOW_WEAK_SIGNATURE_ALG=1, MD5 came to my mind, and got astray.

The scratch build with your patch are at
https://brewweb.devel.redhat.com/taskinfo?taskID=6021487

I installed them in a shared VM in Mountain View, it's IP address is 10.14.1.243.
You can get to it via ssh test.1.243 or as root. Both root and test have password redhat. I have set it up to bbot as single mode where we can change the parameter to kernel from fips-1 to fips=0 as we need. I'll test tomorrow when I get to the office.

Created attachment 771970 [details]
Enable FIPS mode if the system FIPS mode flag is set

Bob's patch updated to work with the nss-3.15 sources as we now have on rhel-7.

(In reply to Bob Relyea from comment #58)
> Kamil: The error you are running into doesn't look like a failure to open
> the database, it looks like a failure to authenticate to the token. NSS fips
> is level-2, which means you must authenticate to the token to do any
> cryptographic operations (other than rng or hashing).

Does this really mean one can't in FIPS mode use NSS e.g. to implement a TLS client without authenticating to $some_token, even when said token is not used to access any private/secret keys, and the only long-term key material ever used are the public keys of the trusted CAs?

Miloslav: Yes, that's part of the level 2 requirements. Actually it's also a level-1 requirement, but usually it's handled by weasel words in the security policy. At level-2 you need to authenticate before you access and CPS (including short term keys).

Also, are we getting audit events when in FIPS mode and the app is running as root? You might see them with aureport --start today --crypto.

(In reply to Bob Relyea from comment #58)
> Try this without the patch:
> modutil -fips true sql:/etc/pki/nssdb
> 
> Then retry your command. If it fails with the same message we are running
> into a curl mismatch in NSS FIPS mode.

Yes, it fails with the same message.

> Does this really mean one can't in FIPS mode use NSS e.g. to implement a TLS
> client without authenticating to $some_token, even when said token is not 
> used to access any private/secret keys, and the only long-term key material
> ever used are the public keys of the trusted CAs?

I should be clear about this. NSS itself does not have a 'FIPS mode'. NSS proper doesn't actually do crypto, it's a big PKCS #11 switch which selects a token to do crypto in.

FIPS is a characteristic of softoken (the default token for NSS). When not in FIPS mode, softoken provides to different tokens for use: 1) a read only crypto token that has no storage and does not require authentication to use. and 2) a database token which may or may not require authentication to use (depending on the state of the key database). In FIPS mode these tokens are combined, so to use crypto you need to first authenticate. 

bob

(In reply to Bob Relyea from comment #70)
> > Does this really mean one can't in FIPS mode use NSS e.g. to implement a TLS
> > client without authenticating to $some_token, even when said token is not 
> > used to access any private/secret keys, and the only long-term key material
> > ever used are the public keys of the trusted CAs?
> 
> I should be clear about this. NSS itself does not have a 'FIPS mode'. NSS
> proper doesn't actually do crypto, it's a big PKCS #11 switch which selects
> a token to do crypto in.

Right now, with the FIPS mode of NSS's softoken independent, we are fine; but if the softoken started respecting /proc/sys/crypto/fips_enabled as this bug requests, we'd end up with the difficulties nentioned in comments #61 and #65.

I could see the fips flag enabling level 1 support by default. That would be highly desirable and would make nss work like the rest of the crypto libraries. But I think level 2 support should be distinct and differently enabled than the fips flag to prevent systemic problems.

Created attachment 781788 [details]
Allow level1 fips mode if the db has no password.

This should allow level 1 FIPS operation in softoken.

The patch for nss breaks SSL when the system is in FIPS mode.  Installing nss-softokn with the second patch applied does not make any difference.  I also noticed that the softokn FIPS flag cannot be flipped to false any more with the patched nss:

# modutil -fips false -dbdir sql:/etc/pki/nssdb

WARNING: Performing this operation while the browser is running could cause
corruption of your security databases. If the browser is currently running,
you should exit browser before continuing this operation. Type
'q <enter>' to abort, or <enter> to continue:

PKCS #11 module could not be removed because it is still in use.
ERROR: Unable to switch FIPS modes.

remind myself I need to comment after investigating...

> I also noticed that the softokn FIPS flag cannot be flipped to false any more with the patched nss:
> 
> # modutil -fips false -dbdir sql:/etc/pki/nssdb
> 
> WARNING: Performing this operation while the browser is running could cause
> corruption of your security databases. If the browser is currently running,
> you should exit browser before continuing this operation. Type
> 'q <enter>' to abort, or <enter> to continue:
> 
> PKCS #11 module could not be removed because it is still in use.
> ERROR: Unable to switch FIPS modes.

Were you running in fips=1 mode when this happened? If so that's on purpose, You can't switch out of FIPS mode manually when the system fips flag is on. Applications like firefox will actually gray out their 'turn fips off' flag.

If you are in fips=0, however, this would be a bug in the patch.

bob

(In reply to Bob Relyea from comment #77)
> Were you running in fips=1 mode when this happened? If so that's on purpose,
> You can't switch out of FIPS mode manually when the system fips flag is on.
> Applications like firefox will actually gray out their 'turn fips off' flag.
> 
> If you are in fips=0, however, this would be a bug in the patch.

Sorry, but after two months, I do not remember the details any more.  I would need to give it another try.  Do you have up2date fixed packages for testing?

Kamil,

Yes, I'll attach it soon.

bob

Created attachment 811443 [details]
Version 2: Allow level1 fips mode if the db has no password.

This should allow level 1 operations if the primary database doesn't have a password.

(NOTE: we should also figure out how to supply a password in curl to support level-2 tokens, including softoken in level-2 mode, but that's not the subject of this bug.

I've tested with both patches and I can switch between fips and non-fips in the databases if the kernel fips=0. I shouldn't be able to switch modes if the kernel fips=1.

bob

Output of attachment #662399 [details] after booting with fips=0:

# sh nss-fips.c
++ pkg-config nss --cflags --libs
+ gcc nss-fips.c -I/usr/include/nss3 -I/usr/include/nspr4 -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl
+ setup-nsssysinit.sh status
NSS sysinit is disabled
+ KFIPS=/proc/sys/crypto/fips_enabled
+ printf '%s = %s\n' /proc/sys/crypto/fips_enabled 0
/proc/sys/crypto/fips_enabled = 0
+ ./a.out sql:/etc/pki/nssdb
NSS_Init("sql:/etc/pki/nssdb") = 0
PK11_IsFIPS() = 0

++ mktemp
+ PASSWD=/tmp/tmp.OxiMEVJH9j
++ mktemp -d
+ DB=/tmp/tmp.LGyqT1bzHz
+ certutil -N -d /tmp/tmp.LGyqT1bzHz -f /tmp/tmp.OxiMEVJH9j
password file contains no data
+ echo
+ modutil -dbdir /tmp/tmp.LGyqT1bzHz -fips true
+ ./a.out /tmp/tmp.LGyqT1bzHz
NSS_Init("/tmp/tmp.LGyqT1bzHz") = 0
PK11_IsFIPS() = 1

+ ./a.out sql://tmp/tmp.LGyqT1bzHz
NSS_Init("sql://tmp/tmp.LGyqT1bzHz") = 0
PK11_IsFIPS() = 0

+ exit 0

Output of attachment #662399 [details] after booting with fips=1:

# sh nss-fips.c
++ pkg-config nss --cflags --libs
+ gcc nss-fips.c -I/usr/include/nss3 -I/usr/include/nspr4 -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl
+ setup-nsssysinit.sh status
NSS sysinit is disabled
+ KFIPS=/proc/sys/crypto/fips_enabled
+ printf '%s = %s\n' /proc/sys/crypto/fips_enabled 1
/proc/sys/crypto/fips_enabled = 1
+ ./a.out sql:/etc/pki/nssdb
NSS_Init("sql:/etc/pki/nssdb") = 0
PK11_IsFIPS() = 1

++ mktemp
+ PASSWD=/tmp/tmp.RkkeCRHhJG
++ mktemp -d
+ DB=/tmp/tmp.m2PwFC6NWB
+ certutil -N -d /tmp/tmp.m2PwFC6NWB -f /tmp/tmp.RkkeCRHhJG
password file contains no data
+ echo
+ modutil -dbdir /tmp/tmp.m2PwFC6NWB -fips true
FIPS mode already enabled.
+ ./a.out /tmp/tmp.m2PwFC6NWB
NSS_Init("/tmp/tmp.m2PwFC6NWB") = 0
PK11_IsFIPS() = 1

+ ./a.out sql://tmp/tmp.m2PwFC6NWB
NSS_Init("sql://tmp/tmp.m2PwFC6NWB") = 0
PK11_IsFIPS() = 1

+ exit 0

(In reply to Bob Relyea from comment #80)
> (NOTE: we should also figure out how to supply a password in curl to support
> level-2 tokens, including softoken in level-2 mode, but that's not the
> subject of this bug.

If the NSS certificate database is protected by a password, it can be supplied by CURLOPT_KEYPASSWD.  libcurl passes the string to NSS via PK11_SetPasswordFunc() and SSL_SetPKCS11PinArg() in order to be able to open the database.

Thanks Kamil,

Everything looks right now, does curl run normally when there is not password on the database and FIPS is enabled now?

elio, I think we can put these in. One patch is to nss-softokn, so you'll need an additional bug.

bob

(In reply to Bob Relyea from comment #86)
> Everything looks right now, does curl run normally when there is not
> password on the database and FIPS is enabled now?

Basic SSL connections appear to work normally in FIPS mode.  I did not check, whether the change affects the available cipher-suites.  And I still have to look at the other protocols using MD5 and DES to check whether they are also affected by this change.

NTLM authentication (based on MD5 and DES) breaks in FIPS, which is expected.  Allowed SSL cipher-suites do not seem to be affected by this change.

Moreover, I see another suspicious thing - assume that (sql:/)/etc/pki/nssdb is in fips "state" (set via modutil). 

a) nss-sysinit is enabled, system is rebooted with fips=0 kernel flag:

   * PK11_IsFIPS() for /etc/pki/nssdb returns 1
   * PK11_IsFIPS() for sql://etc/pki/nssdb returns 0

   How is this possible? Shouldn't both values be set to 0?

b) nss-sysinit is disabled, system is rebooted with fips=0 kernel flag

   * PK11_IsFIPS() for /etc/pki/nssdb returns 1
   * PK11_IsFIPS() for sql://etc/pki/nssdb returns 1

   Why is that? This is quite confusing. 

Does nss-sysinit play any role in fips "state" of /etc/pki/nssdb? I do not see it documented, but it looks like it does (see above).

And finally, is there a RHEL6 variant of this bug? Shouldn't it be?

It looks like to hand point the system DB into fips mode.

(run 'modutil -list -db /etc/pki/nssdb' and 'modutile -list -db sql:/etc/pki/nssdb' as root, or without sysinit).

nss-sysinit only works with sql: databases, which is why you get different results between the two when it is installed. (most likely it's looking at your per/user database for the FIPS indication, not the system library).


Setting the fips flag on the database is how individual applications turn on fips mode (like the browser, which has a FIPS button). We still allow applications to independently turn on FIPS mode just as they've always done (even when the system isn't in FIPS mode). What we've changed is that NSS looks at the system flag and forces FIPS mode when the system is in FIPS mode. Those applications that know about FIPS mode are told that they are in FIPS mode and that they can't switch out of FIPS mode (The browser button to switch out of FIPS mode will be grayed out when the system is in FIPS mode).

In RHEL6 we only have the traditional NSS FIPs mode setting, not the 'force FIPS on system' setting.

(In reply to Bob Relyea from comment #97)
> It looks like to hand point the system DB into fips mode.
> 
> (run 'modutil -list -db /etc/pki/nssdb' and 'modutile -list -db
> sql:/etc/pki/nssdb' as root, or without sysinit).
> 

Well, ok it turns out it works fine then.

> nss-sysinit only works with sql: databases, which is why you get different
> results between the two when it is installed. (most likely it's looking at
> your per/user database for the FIPS indication, not the system library).
> 
> 
> Setting the fips flag on the database is how individual applications turn on
> fips mode (like the browser, which has a FIPS button). We still allow
> applications to independently turn on FIPS mode just as they've always done
> (even when the system isn't in FIPS mode). What we've changed is that NSS
> looks at the system flag and forces FIPS mode when the system is in FIPS
> mode. Those applications that know about FIPS mode are told that they are in
> FIPS mode and that they can't switch out of FIPS mode (The browser button to
> switch out of FIPS mode will be grayed out when the system is in FIPS mode).
> 
> In RHEL6 we only have the traditional NSS FIPs mode setting, not the 'force
> FIPS on system' setting.

Thanks for clarifying this. Aren't there any plans to fix this in 6.6? I would say it might be needed for FIPS recert., wouldn't it? 

BTW: What about Comment #93 and #94? Once that will be resolved I thing we can move forward and possibly close this.

> Thanks for clarifying this. Aren't there any plans to fix this in 6.6? 
> I would say it might be needed for FIPS recert., wouldn't it? 

There are no plans to change this in 6.6, though the need for a FIPS recert isn't the reason (the change is in NSS proper, not in softoken itself). The main reason for this is potential regressions in customer applications. We found a few in ours, which we either worked around or fixed, so it would be a relatively risky change for RHEL 6.

bob

I see, I see. Thank Bob.

Can anyone look on those missing hmac files? (See "BTW" in Comment #98).

Created attachment 852443 [details]
specfile changes to appy the patch -- in patch form

(In reply to Elio Maldonado Batiz from comment #101)
> Created attachment 852443 [details]
> specfile changes to appy the patch -- in patch form

Aren't those specfile changes related to different bug (Bug 1020395)? 

The most important issue right now is that no hmac files are included in nss packages. How can be integrity of modules verified? I am boosting the severity because of this.

(In reply to Ondrej Moriš from comment #102)
> (In reply to Elio Maldonado Batiz from comment #101)
> > Created attachment 852443 [details]
> > specfile changes to appy the patch -- in patch form
> 
> Aren't those specfile changes related to different bug (Bug 1020395)? 

You are correct. As a matter or fact the patch has already been applied in nss-softokn whre it is actually neeeded.

$ grep -i patch11 nss-softokn.spec 
Patch11:           nss-softokn-allow-level1.patch
%patch11 -p0 -b .allow_level1

Applying it here doesn't add anything of value, expect for consistency when reading code, as the softoken/freebl source directories are later deleted from the source tree. We don't want to compile them again as they have already been build and tested as part of the nss-softokn build. That was bug 689919.

> 
> The most important issue right now is that no hmac files are included in nss
> packages. How can be integrity of modules verified? I am boosting the
> severity because of this.

No hmac files need to be inculed bey nss (the nss package) since they are meant to be included by nss-softokn package. Verification of the integrity of the module is handled in the nss-softokn package, that's the one with the crypto boundary lies. That's part of the work to conform to new FIPS standard, see Bug 1004102.  It was to facilitate handling, and preserving, FIPS validation that we split off the code within the cryptographic boundary into a separate package three or so years ago.

(In reply to Elio Maldonado Batiz from comment #103)
> (In reply to Ondrej Moriš from comment #102)
> > The most important issue right now is that no hmac files are included in nss
> > packages. How can be integrity of modules verified? I am boosting the
> > severity because of this.
> 
> No hmac files need to be inculed bey nss (the nss package) since they are
> meant to be included by nss-softokn package. Verification of the integrity
> of the module is handled in the nss-softokn package, that's the one with the
> crypto boundary lies. That's part of the work to conform to new FIPS
> standard, see Bug 1004102.  It was to facilitate handling, and preserving,
> FIPS validation that we split off the code within the cryptographic boundary
> into a separate package three or so years ago.

Ah yes, I am aware of this, I wrote it incorrectly. I means that I see no hmac files in nss* packages:

# # rpm -qa | grep "^ns"
nspr-4.10.2-4.el7.x86_64
nss-3.15.4-4.el7.x86_64
nss_compat_ossl-0.9.6-8.el7.x86_64
nss-util-3.15.4-2.el7.x86_64
nss-tools-3.15.4-4.el7.x86_64
nss-sysinit-3.15.4-4.el7.x86_64
nss-softokn-3.15.4-2.el7.x86_64
nss-util-devel-3.15.4-2.el7.x86_64
nss-softokn-freebl-3.15.4-2.el7.x86_64
nspr-devel-4.10.2-4.el7.x86_64
# for X in $(rpm -qa | grep "^ns"); do rpm -ql $X; done | grep mac
#

From what I remember, in RHEL-6.5 timeframe, we moved hmac files to *-fips subpackage, it was then decided to drop it. Isn't it possible that hmac files were mistakenly dropped with it?

My point is - are POST performed? If so, how is possible that POST do not fail without hmac files?