Bug 546221

Summary: firefox doesn't use system certificate store
Product: [Fedora] Fedora Reporter: David Woodhouse <dwmw2>
Component: xulrunnerAssignee: Kai Engert (:kaie) <kengert>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 15CC: amarecek, anton, emaldona, gecko-bugs-nobody, herrold, ionut, johnp, jrb, kajtzu, kdudka, kengert, ktdreyer, kwizart, myllynen, rob.townley, rrelyea, samuel-rhbugs, stransky, tmraz, tomasz.kepczynski, walters
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=449498
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-07 15:33:49 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 466626    
Attachments:
Description Flags
patch
rrelyea: review+
firefox patch
none
evolution patch
none
xulrunner patch to make firefox use system nssdb none

Description David Woodhouse 2009-12-10 08:37:00 EST
There is a system-wide NSS db in /etc/pki/nssdb. I have added my company's internal CA certificates there.

However, firefox still doesn't trust our internal web sites. It doesn't seem to be using the system database.
Comment 1 Martin Stransky 2009-12-10 08:59:20 EST
I don't believe it's a firefox bug...
Comment 2 David Woodhouse 2009-12-12 04:57:57 EST
I tried installing the nss-sysinit package and installing certs with 'certutil -d /etc/pki/nssdb'. But it doesn't seem to make any difference -- neither Evolution nor Firefox seem to know anything about these certificates.

/proc/$PID/maps seems to suggest that they don't have /usr/lib64/libnsssysinit.so mapped.
Comment 3 David Woodhouse 2009-12-13 09:18:11 EST
Test procedure... First we fetch a signing cert (just an example; it doesn't matter which it is), import it into a new application-specific NSS DB, and it works. We remove it from the app's DB, and it doesn't. 
All is well so far...

[root@macbook dwmw2]# curl -k https://www.cacert.org/certs/root.crt > cacert.crt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
102  2569  102  2569    0     0  10740      0 --:--:-- --:--:-- --:--:-- 51380
[root@macbook dwmw2]# mkdir /tmp/nssdb
[root@macbook dwmw2]# certutil -d /tmp/nssdb -t TC,TC,TC -E -i cacert.crt -n cacert
[root@macbook dwmw2]# /usr/lib64/nss/unsupported-tools/tstclnt -d /tmp/nssdb -h www.cacert.org -p 443
subject DN: E=support@cacert.org,CN=www.cacert.org,O=CAcert Inc.,L=Sydney,ST=NSW,C=AU
issuer  DN: E=support@cacert.org,CN=CA Cert Signing Authority,OU=http://www.cacert.org,O=Root CA
0 cache hits; 1 cache misses, 0 cache not reusable
0 stateless resumes
^C
[root@macbook dwmw2]# certutil -d /tmp/nssdb -D -n cacert
[root@macbook dwmw2]# /usr/lib64/nss/unsupported-tools/tstclnt -d /tmp/nssdb -h www.cacert.org -p 443
tstclnt: read from socket failed: Peer's Certificate issuer is not recognized.
Comment 4 David Woodhouse 2009-12-13 09:26:31 EST
And this shows the failure... this one ought to _work_, surely?

[root@macbook dwmw2]# setup-nsssysinit.sh on
[root@macbook dwmw2]# certutil -d /etc/pki/nssdb -t TC,TC,TC -E -i cacert.crt -n cacert
[root@macbook dwmw2]# /usr/lib64/nss/unsupported-tools/tstclnt -d /tmp/nssdb -h www.cacert.org -p 443
tstclnt: read from socket failed: Peer's Certificate issuer is not recognized.

The issuer _should_ be recognised -- I just added it to the system database!
It's not just tstclnt that fails; evolution and firefox fail too.

curl does work, but I think that's because it actually uses /etc/pki/nssdb as its "application" database.
Comment 5 Kamil Dudka 2009-12-13 09:30:31 EST
The behavior looks slightly similar to bug 545779. Could you please try the patch from there (including the change from comm. #20)?
Comment 6 David Woodhouse 2009-12-13 09:55:50 EST
You mean just the patch to nsssysinit.c in comm. #18, with the extra one-liner?
I built that and installed the resulting libnsssysinit.so library. But when I run 'tstclnt' as described, the atime on the library doesn't change -- it isn't even being loaded. The atime on /etc/pki/nssdb/pkcs11.txt doesn't change either.

Are you able to reproduce the problem using the commands given above? It should be fairly simple.
Comment 7 David Woodhouse 2009-12-13 11:32:41 EST
I've been playing with this, and reading the documentation at
https://wiki.mozilla.org/NSS_Shared_DB_And_LINUX

Apparently, you're supposed to initialise NSS with /etc/pki/nssdb specified as the database directory (which firefox and evolution aren't doing), and then the pkcs11.txt file there is supposed to magically add your ~/.pki/nssdb directory too.

That mostly does actually seem to work, with the caveat that I have to run 'setup-nsssysinit.sh off' before using certutil to add certs to /etc/pki/nssdb, otherwise they end up in /root/.pki/nssdb which is a little suboptimal.

As a normal user, I have to run 'certutil -d ~/.pki/nssdb -N' before I can import any _personal_ certs to my own store. And once I've done that, 'certutil -L -d /etc/pki/nssdb' doesn't show any of the system-wide certs. But they _do_ work.

So, I think the biggest problem is that the various tools aren't using sql:/etc/pki/nssdb as their database directory as they should? They'll probably want some kind of migration tool too.
Comment 8 Elio Maldonado Batiz 2009-12-13 13:34:41 EST
Bob wrote this when we where testing system nss back in 3.12.4. There written by Bob and I have tried to update them for 3.12.5 and must warn you that there may be some errors. Part 2 should be what you need to trick Firefox/Thunderbird into using the the sahred database.

Part 1: Enabling system NSS (for 3.12.4 not needed for 3.12.5)

1) As root examine /etc/pki/nssb. If the databases are legacy (secmod.db, key3.db and cert8.db) but no shared db's (pkcs11.txt, key4.db and cer8.db) you the need steps 2, 3 and 4.

2) As root run certutil -K -X -d sql:/etc/pki/nssdb  (this will create sql db's
from the local dbm database).

3) as yourself, in your .bashrc or .cshrc (or whatever shell you use) add:
     export NSS_DEFAULT_DB_TYPE=sql
[ for csh/tcsh that sould be setenv 'NSS_DEFAULT_DB_TYPE sql' ]

4) as yourself (not as root!!!) execute 'certutil -N -d sql:/etc/pki/nssdb'. If
you supply a password, this password will eventually become your new
firefox/thunderbird master password.

Part 2: Convince Firefox, Seamonkey, Xulrunner and Thunderbird to use
system NSS. Firefox and Thunderbird still use private directories to
store their NSS databases. It's possible, however, to convince them to
open system NSS. All these steps should be performed as a user. In the
future these applications would handle these steps by themselves
automatically.

Do each of the following steps in each of the profile directories for
which you wish to use system NSS in. Firefox, thunderbird, and seamonkey
each have their own directories and can have multiple profiles (if you
don't know what a profile is, you probably only have one per
application). You can the base of the profile directories for firefox at
~/.mozilla/firefox and thunderbird at ~/.thunderbird. In that base
directory there is a file called 'profiles.ini' which lists all the
profiles that are known for that application. For each profile there's a
line called 'Path=' which points to the actual profile directory.
Usually it's a subdirectory under the current directory and has a random
'salted' name like 'quxz7me5.default'. do the following instructions
while cd'd to that directory:

1) certutil -K -X -d sql:.
    (if you have a master password set, You'll have to provide it here).
This will create a new sql database from your old dbm database.

2) certutil --merge -d sql:~/.pki/nssdb --source-dir sql:.
    (if you have a password set on sql:~/.pki/nssdb (from step 4 above),
You'll have to provide it here. If that password is different from the
master password for the profile, you'll also have to supply the profile
password. If ~/.pki/nssdb has to password, but the profile database
does, ~/.pki/nssdb will inherit that password (which you will then need
on future instances of this step). If you want to change that password
use 'certutil -N -d sql:~/.pki/nssdb'. Supplying an empty password will
remove any password.

2) edit pkcs11.txt
    2a) in the 'NSS Internal PKCS #11 Module' stanza. change:
    
parameters=configdir='/home/{you}/.mozilla/firefox/quxy6me8.default'
..... Flags=internal,critical
       to
     parameters="configdir='sql:/etc/pki/nssdb' .....
Flags=internal,critical,moduleDBOnly"
    NOTE: the ... represents a very long string of parameters that
should remain the same. Note the addition of the double quotes around
the parameters. This tells NSS to use the parameters as given and now
rebuild them based on application input.
    2b) save pkcs11.txt

3) Make sure you set NSS_DEFAULT_DB_TYPE to sql in your environment
before you start FF or TB

Restoring your system to the old databases:
Simply unset the NSS_DEFAULT_DB_TYPE variable and all your applications
will fall back to their former dbm style database
------------------------------------------------
Comment 9 David Woodhouse 2009-12-13 14:09:37 EST
That doesn't seem to work. It adds the database specified by getUserDb(), and then it the database specified by the app (~/.mozilla/firefox/...) in read-only mode, with a comment saying 'now the system database'.

It never actually _does_ add the db specified by getSystemDb().

If I change the get_list() function in nssysinit.c to ignore what the app is asking for and just initialise the userdb and the sysdb, then everything works as expected.

This is my pkcs11.txt file from my firefox config directory:

[dwmw2@macbook b8v9tyu0.default]$ cat pkcs11.txt 
library=libnsssysinit.so
name=NSS Internal PKCS #11 Module
parameters=configdir='sql:/etc/pki/nssdb'  certPrefix='' keyPrefix='' secmod='secmod.db' flags= updatedir='' updateCertPrefix='' updateKeyPrefix='' updateid='' updateTokenDescription='' 
NSS=Flags=internal,moduleDBOnly,critical trustOrder=75 cipherOrder=100 slotParams=(1={slotFlags=[RSA,DSA,DH,RC2,RC4,DES,RANDOM,SHA1,MD5,MD2,SSL,TLS,AES,Camellia,SEED,SHA256,SHA512] askpw=any timeout=30})
Comment 10 David Woodhouse 2009-12-13 14:14:15 EST
Created attachment 378063 [details]
patch

Working patch (including the patch from bug #545799). Note:


    if (filename && 0 /* This doesn't actually work. If we register
			 both this and the sysdb (in either order)
			 then only one of them actually shows up */) {
Comment 11 David Woodhouse 2009-12-13 20:01:24 EST
Created attachment 378124 [details]
firefox patch

We need a patch along these lines in firefox to make it use sql:/etc/pki/nssdb rather than initialising its own database. The instructions on the wiki are all very well for testing, but this ought to be the default behaviour (if nss-sysinit is installed). Untested, and maybe it ought to fall back to the profile directory if /etc/pki/nssdb fails.
Comment 12 David Woodhouse 2009-12-13 20:19:25 EST
Created attachment 378127 [details]
evolution patch

And something like this should do it for evolution-data-server.
Comment 13 Bob Relyea 2009-12-14 14:48:26 EST
Comment on attachment 378063 [details]
patch

r+

I need to look into why the additional directory is not getting initialized (there should be no limit to adding more directories here.
Comment 14 Bob Relyea 2009-12-14 15:04:32 EST
RE the firefox and evolution patches...

These patches just move to the sql light DB's without moving the contents of the old DB's. This can be done with certutil by hand using the --merge or the --update-merge flag, or it can be done in the application itself. There's an NSS init command which will automatically check to see if an old database has been merged, and if not initiate the steps to merge that database. If the old database has already been merged, or does not exist, the init call functions just like any other NSS init call.

Instructions for how do do this is available here: https://wiki.mozilla.org/NSS_Shared_DB#Mode_3A Any upstream available patch should probably include some of this code, though for our uses, it's certainly cleaner just to switch, giving the users a set of instructions on how to use certutil. For most users, the biggest loss they will feel after if you just switch to sqlite is the loss of their web passwords, which are encrypted with a key stored in their FF database.

Thanks you very much for creating both this bug and these patches David!

bob
Comment 15 David Woodhouse 2009-12-15 15:12:50 EST
(In reply to comment #14)
> Instructions for how do do this is available here:
> https://wiki.mozilla.org/NSS_Shared_DB#Mode_3A 

I filed https://bugzilla.gnome.org/show_bug.cgi?id=604644 for Evolution. Is it wrong to assume that someone's already looking at doing the right thing for firefox, so I don't need to file that in mozilla bugzilla?

Just to confirm: If the applications are fixed to do the right thing, then my patch in comment #10 is superfluous, right?

I'll open a separate NSS bug for the fact that root can't add certs to sql:/etc/pki/nssdb with certutil (because the new certs end up in /root/.pki/nssdb instead). And reassign this bug to firefox where I believe it belongs.

(should we dupe for xulrunner/tbird/etc.?)
Comment 16 Fedora Update System 2010-01-07 01:35:10 EST
nss-3.12.5-2.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/nss-3.12.5-2.fc12
Comment 17 Elio Maldonado Batiz 2010-01-07 02:23:08 EST
(In reply to comment #16) Mistakenly included this bug in nss-3.12.5-2.fc12 bugs fixed list. The r+'ed patch was duly included for its short term benefit. I realize that this bug's request is for Firefox itself to use the system wide system database.
Comment 18 David Woodhouse 2010-01-11 09:50:04 EST
We should drop my patch in comment #10 -- it's a dirty hack so that we can trick legacy apps into using the system database, and it's unlikely to be acceptable upstream. We should just fix our apps so that they use the system database properly.

Things are confusing and broken enough already, without adding to it.
Comment 19 Kai Engert (:kaie) 2010-03-23 15:14:49 EDT
I'm not clear on the earlier comments in this bug.

If I understand correctly, this bug requests that Firefox should use the root CAs that are installed in the global system location /etc/pki/nssdb

However, a Firefox profile clearly shouldn't use that global database for all certificate that NSS needs to manage, in other words, Firefox still needs a database that is private to the user, whether it's a user-global database (using shared db) or a profile-local database (which is still the default of Firefox).

So, I believe this bug requests some dynamic merge. It probably requests that Firefox should continue to use an NSS database from /home, and merge it with the root CA list stored in /etc

I believe we don't have such a feature yet.
If we want this to happen, it must be implemented upstream, in the Mozilla platform core code.

We'd have to define how the merge shall happen. Both the global /etc and the /home database can contain information regarding to roots. For example, a user can disable trust from builtin roots.

It will require to define the order of preference for conficting trust settings for a single CA.
Comment 20 David Woodhouse 2010-03-23 17:15:56 EDT
(In reply to comment #19)
> So, I believe this bug requests some dynamic merge. It probably requests that
> Firefox should continue to use an NSS database from /home, and merge it with
> the root CA list stored in /etc
> 
> I believe we don't have such a feature yet.
> If we want this to happen, it must be implemented upstream, in the Mozilla
> platform core code.

It _has_ been implemented in upstream NSS already. That's the whole point in the 'shared db' stuff.
Comment 21 Elio Maldonado Batiz 2010-03-23 18:31:21 EDT
The request is for Firefox to open the system-wide certificate store as well as its own. That is, take advantage of the nss-sysinit module when available in the the OS - only Fedora at the moment. It would first have to become shared db aware which it isn't yet AFAIK. I guess a review of the nss-sysinit design document at https://wiki.mozilla.org/NSS_Shared_DB_And_LINUX may help clarify things a bit.
Comment 22 David Woodhouse 2010-06-11 11:03:06 EDT
Now fixed in Evolution 2.30 branch and HEAD
Comment 23 David Woodhouse 2010-06-12 06:48:56 EDT
(In reply to comment #22)
> Now fixed in Evolution 2.30 branch and HEAD    

... but see bug 603313.
Comment 24 David Woodhouse 2010-06-15 14:31:00 EDT
Created attachment 424256 [details]
xulrunner patch to make firefox use system nssdb

With the NSS patch I recently added to bug 603313, and this xulrunner patch, I now have firefox properly using the system NSS database (importing the contents of the old DBM database in the profile directory).

If the system database isn't enabled, it should just continue to use the old DBM database.
Comment 25 David Woodhouse 2010-06-15 15:26:34 EDT
This is all basically working now. We should be able to do a Fedora 14 feature for using the system database, and then we can fix bug 466626 (as part of the feature)
Comment 26 Ken Dreyer 2010-07-02 10:52:28 EDT
Can someone add Mozilla bug 449498 under "External Bugs". I don't have permissions.
Comment 27 David Woodhouse 2010-07-02 11:13:45 EDT
See also the discussion in https://bugzilla.mozilla.org/show_bug.cgi?id=490238#c37
Comment 28 Elio Maldonado Batiz 2010-08-03 18:35:52 EDT
This reflects portability-oriented modifications I made to your own patch to 
have it as an NSS patch whic is desirable. It's sleeping in a samples bug where
it doesn't belong and gets ignored. I could submit it as an NSS bug fix. 
The other option would be for the xulrunner-only fix in which case we could
make similar portability modifications to your patch here in attachment 424256 [details]. 
There is something to be said to having in NSS proper for all applications to benefit. Some might argue for the more conservative road of just patching here for
xulrunner, at least initially. Any preferences?
Comment 29 David Woodhouse 2010-08-03 18:46:35 EDT
Other applications (chrome, evolution, etc.) are already doing the check for themselves, and opening the system database or the user's database accordingly. I think we might as well do the same in xulrunner.

It'll be nice when NSS makes it easier, but I don't think we want to require a new version of NSS before this finally starts to work.
Comment 30 Fedora Admin XMLRPC Client 2010-09-07 16:54:23 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 31 Kaj J. Niemi 2010-09-12 17:42:03 EDT
I stumbled on the same for RHEL6 beta 2. It would be very nice to have all users use the system wide certificate store instead of their own for verifying private enterprise CA certificates.
Comment 32 Elio Maldonado Batiz 2010-09-12 18:46:30 EDT
I agree, the nss part is done so I have changed the component to xulrunner where it now properly belongs.
Comment 33 Elio Maldonado Batiz 2010-09-12 19:58:52 EDT
The patch as it is works in fedora but one should consider portability as eventually it needs to be submitted upstream to mozilla. To start I would suggest using the NSPR types and NSPR functions.
Adding 
+#include "prio.h" to pickup the NSPR functions
using PRBool instead of int for the boolean variable, and replacing the use of ::fopen, ::fgets, ::strcomp, and ::flose with PR_Open, PR_Read, PORT_Strcmp, and PR_Close

It would look like this:
+#define NSS_SYSTEM_DB "/etc/pki/nssdb"
+    SECStatus init_rv = SECFailure;
+    const char *nssdb = profileStr.get();
+
+    FILE *f = PR_Open(NSS_DEFAULT_SYSTEM "/pkcs11.txt", PR_RDONLY, 0);
+    if (f) {
+        PRBool found = PR_FALSE;
+        char buf[80];
+        /* Check whether the system NSS db is actually enabled */
+        while (PR_Read(f, buf, 80) && !found) {
+          if (!PORT_Strcmp(buf, "library=libnsssysinit.so\n"))
+            found = PR_TRUE;
+        }
+        PR_Close(f);
+        if (found) {
+          nssdb = "sql:"NSS_SYSTEM_DB;
+          init_rv = ::NSS_InitWithMerge(nssdb,
+                                        "", "", SECMOD_DB,
+                                        profileStr.get(), "", "",
+                                        profileStr.get(), profileStr.get(), 0);
+        }
+    }
+    /* If no sysdb found, or opening it failed, try opening the
+       old db from the profile directory. If that fails, subsequent
+       (readonly, nodb) attempts will be on the system db */
+    if (init_rv == SECFailure)
+      init_rv = ::NSS_InitReadWrite(profileStr.get());
The NSS_DEFAULT_SYSTEM is a unix/linux path so a function like find_default_system_db_dir() may be needed to be even more cross-platform.
Comment 34 Bug Zapper 2010-11-03 23:43:14 EDT
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 35 David Woodhouse 2010-11-03 23:53:47 EDT
Still not fixed in F14 -- I'm still having to build my own xulrunner package to fix it.
Comment 36 Martin Stransky 2010-11-04 11:12:28 EDT
We need to move this patch upstream.
Comment 37 David Woodhouse 2010-11-29 09:58:30 EST
Gr, yet another Fedora xulrunner update breaks this for me; yet another local rebuild with the required patch to fix it again...
Comment 38 Elio Maldonado Batiz 2010-11-29 14:18:45 EST
I was hoping the patch could receive review downstream in Fedora before moving upstream. For upstream submission it would be for the Personal Security Manager, see https://wiki.mozilla.org/Modules/All#Security_-_Mozilla_PSM_Glue
Comment 39 Martin Stransky 2010-12-20 09:31:55 EST
Filled as https://bugzilla.mozilla.org/show_bug.cgi?id=620373
Comment 40 David Woodhouse 2011-08-15 18:33:48 EDT
Still broken in Fedora 15.
Comment 41 Martin Stransky 2012-05-07 09:13:47 EDT
So what's up with this one? Do we want to use this patch or not? Kai, I think it's your area, right? So It may not be assigned to me. If you'd like to have this patch in Firefox/Fedora, just put it there or ping me...
Comment 42 Fedora End Of Life 2012-08-07 15:33:57 EDT
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping