Bug 1752303

I can confirm this issue (Fedora 30/Firefox 70). It also happens with bugzilla.redhat.com, probably because it uses a DigiCert certificate.

Creating a new Firefox profile did not help. My workaround is to repeatedly launch Firefox. In roughly 20% of cases, I will not get a (persistent) warning.

The specific error reported by Firefox when this occurs for me is MOZILLA_PKIX_ERROR_MITM_DETECTED.

Yes, I do see that too, on bugzilla.mozilla.org page.

Ueno, any idea here?

Would like to add in I'm running Fedora 30 (KDE spin though I don't believe that matters) as well with Firefox version 70 now and it has the same issue when I resume a session with firefox open to trello.com which also uses digicert. The way I was able to fix this for myself for the time being it looks like was to actually go and manually install the Digicert Global Root CA certificate listed by the error from their site and add it within Firefox itself. Couldn't figure out a reason why this specifically was causing such an issue.

(In reply to Martin Stransky from comment #4)
> Ueno, any idea here?

No idea, I'm afraid. But I suspect that the MOZILLA_PKIX_ERROR_MITM_DETECTED error might be a red herring; the actual problem is that sometimes Firefox couldn't find the root cert from the system trust store, something similar to:
https://bugzilla.mozilla.org/show_bug.cgi?id=1530429

Maybe good to ask on the upstream bugzilla (if it is not done).

Happens in Fedora 31 / Firefox 70.0 too when visiting duckduckgo.com. 

Just when I was editing this post, I refreshed the duckduckgo tab. The problem was gone.  What happened?

Something interesting also happened. I am not sure it's related. I did a "dnf update" and a gpg check also failed.

Problem opening package rpmfusion-nonfree-appstream-data-31-1.fc31.noarch.rpm
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED

Now the cert problem is gone, I clean the dnf cache, run "dnf update" again, and gpg check succeeded.

*** Bug 1766895 has been marked as a duplicate of this bug. ***

Created attachment 1630467 [details]
nss log

Ueno, there's a nss log attached. Can you please briefly check that if it's something Fedora specific/related to our system nss or if we should report that upstream? Thanks.

Created attachment 1630507 [details]
p11-kit log

I just ran env P11_KIT_DEBUG=all /usr/bin/firefox >& debug.log and reproduced the error.

See log.

Created attachment 1630515 [details]
requested log with P11_KIT_DEBUG=trust

Thank you for the logs, but I still can't tell where is the problem.

After comparing the p11-kit logs between the successful case and the unsuccessful case, the difference seems to be the following queries missing in the latter case (in terms of DigiCert SHA2 Secure Server CA):

  sys_C_FindObjectsInit: in: 18, (4) [ { CKA_TOKEN = (1) "\x01" }, { CKA_CLASS = CKO_CERTIFICATE }, { CKA_ISSUER = (79) "0M1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x150\x13\x06\x03U\x04\n\x13\x0cDigiCert Inc1'0%\x06\x03U\x04\x03\x13\x1eDigiCert SHA2 Secure Server CA" }, { CKA_SERIAL_NUMBER = (18) "\x02\x10\x0c\xa9\xc6Ca\xbf\xc9*y\xb1\xdd\x9c\xfb\x9eH\xec" } ]
  sys_C_FindObjectsInit: in: 18, (4) [ { CKA_TOKEN = (1) "\x01" }, { CKA_CLASS = CKO_CERTIFICATE }, { CKA_ISSUER = (79) "0M1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x150\x13\x06\x03U\x04\n\x13\x0cDigiCert Inc1'0%\x06\x03U\x04\x03\x13\x1eDigiCert SHA2 Secure Server CA" }, { CKA_SERIAL_NUMBER = (16) "\x0c\xa9\xc6Ca\xbf\xc9*y\xb1\xdd\x9c\xfb\x9eH\xec" } ]
  sys_C_FindObjectsInit: in: 18, (4) [ { CKA_TOKEN = (1) "\x01" }, { CKA_CLASS = CKO_NSS_TRUST }, { CKA_ISSUER = (79) "0M1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x150\x13\x06\x03U\x04\n\x13\x0cDigiCert Inc1'0%\x06\x03U\x04\x03\x13\x1eDigiCert SHA2 Secure Server CA" }, { CKA_SERIAL_NUMBER = (16) "\x0c\xa9\xc6Ca\xbf\xc9*y\xb1\xdd\x9c\xfb\x9eH\xec" } ]

while it has a query without CKA_SERIAL_NUMBER:

  sys_C_FindObjectsInit: in: 18, (3) [ { CKA_CERTIFICATE_TYPE = CKC_X_509 }, { CKA_CLASS = CKO_CERTIFICATE }, { CKA_SUBJECT = (79) "0M1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x150\x13\x06\x03U\x04\n\x13\x0cDigiCert Inc1'0%\x06\x03U\x04\x03\x13\x1eDigiCert SHA2 Secure Server CA" } ]

I suppose the next step would be to check how/where these queries are constructed, which I suggest hearing from upstream.

I'm going to create a test build with in-tree nss to check if that fixes this issue.

(In reply to Martin Stransky from comment #15)
> I'm going to create a test build with in-tree nss to check if that fixes
> this issue.

Added to firefox-70.0-2 builds.

I'm also experiencing this error on firefox-70.0-1.fc31.x86_64 after upgrading to Fedora 31

Sires that randomly fail to load are for example reddit.com, bugzilla.mozilla.org

New test builds are available here:

F30: https://koji.fedoraproject.org/koji/taskinfo?taskID=38697234
F31: https://koji.fedoraproject.org/koji/taskinfo?taskID=38697225

please check them if you see any issue. It uses in-tree nss so you loose nss system certificate integration which causes the bug we see here.

*** Bug 1767688 has been marked as a duplicate of this bug. ***

I guess you may see bug 1766340 with the builds above.

Bug 1648617 may be a dupe.

I'm not a Red Hat user, so apologies if this comment is not helpful, but I thought this might confirm that the issue is not Red Hat-specific. I experienced the same bug on Arch Linux today, in firefox-70.0.1-1. I previously experienced it on 25 October 2019 in firefox-70.0-1. In both cases the error message referred to the DigiCert Global Root CA, did not affect other browsers, and disappeared after restarting Firefox, meaning that I could not reproduce it for further diagnosis. I had not noticed the bug before upgrading to firefox-70.0-1 on 22 October 2019.

See also https://bbs.archlinux.org/viewtopic.php?id=247067

I upgraded to the test-build and I still can reproduce the error.

rpm -q firefox
firefox-70.0-2.fc30.x86_64

Created attachment 1632523 [details]
p11-kit log test-build firefox-70.0-2.fc30.x86_64

Also see: https://github.com/clearlinux/distribution/issues/1006

FWIW I've noticed this bug occasionally breaks Firefox Sync, because accounts.mozilla.com also uses DigiCert. I noticed because my connected devices did not appear under "Send Tab to Device". Of course it's very sporadic and usually works fine, just like with DuckDuckGo. Certainly it has nothing to do with the homepage setting.

(In reply to Michael Catanzaro from comment #27)
> FWIW I've noticed this bug occasionally breaks Firefox Sync, because
> accounts.mozilla.com also uses DigiCert.

Sorry, I meant accounts.firefox.com.

*** Bug 1761161 has been marked as a duplicate of this bug. ***

After reboot I can't reproduce the bug with the test-build firefox-70.0-2.fc30.x86_64. Maybe it didn't load the new version without reboot.

(In reply to Gerald Zehetner from comment #30)
> After reboot I can't reproduce the bug with the test-build
> firefox-70.0-2.fc30.x86_64. Maybe it didn't load the new version without
> reboot.

Okay, moving to nss as it can't be reproduced with Firefox in-tree nss.

NSS recently added two new fields to the certificate DB: https://hg.mozilla.org/projects/nss/rev/52024949df95c7147c4995bb8ddc094027636001

I guess p11-kit needs to be updated to support these fields, much like it was for CKA_NSS_MOZILLA_CA_POLICY.

(In reply to Jan Alexander Steffens from comment #32)
> NSS recently added two new fields to the certificate DB:
> https://hg.mozilla.org/projects/nss/rev/
> 52024949df95c7147c4995bb8ddc094027636001
> 
> I guess p11-kit needs to be updated to support these fields, much like it
> was for CKA_NSS_MOZILLA_CA_POLICY.

Certainly, but I don't think it is related to this bug as there is no released version that "uses" those attributes.

Until yesterday the firefox-70.0-2.fc31.x86_64 worked without problems with DigiCert's CA for a week.
Yesterday I was force to accept exception to access DigiCert CA's sites.
I have many page preloaded (>100) and each time firefox starts, it change the loading sequence of the pages and the CA's. 
I think the problem is related sequence of CA chain loading in firefox maybe it is GC/allocation bug.

I use uBlocker Origin and NoScript add-on in firefox.

(In reply to Lars S. Jensen from comment #34)
> I have many page preloaded (>100) and each time firefox starts, it change
> the loading sequence of the pages and the CA's. 
> I think the problem is related sequence of CA chain loading in firefox

Interesting. What changes based on the order the pages are loaded is the state of your NSS certificate cache. Based on this, I think I might have found a reproducer: remove all DigiCert intermediate certificates from the certificate cache. Normally this should be safe to do, but due to this bug it causes Firefox to break. I have no clue what is going wrong here between NSS and p11-kit, but hopefully this will help any developers unable to reproduce.

Since loading any webpage could affect this, best to first close all tabs except duckduckgo.com and set Firefox to restore previous tabs on startup to ensure a clean environment. Then: go to Preferences -> Privacy & Security -> scroll to the bottom -> View Certificates... -> Authorities. Scroll down to DigiCert Inc. Delete or Distrust every DigiCert certificate that is labeled Software Security Device. (You should *probably* only need to delete one, DigiCert SHA2 Secure Server CA, but deleting them all is what I just tested.) When doing this, be very careful not to Delete or Distrust any certificate labeled Default Trust, because those are trust roots that should not be messed with. The UI here is very confusing, but I believe the Software Security Device label indicates an intermediate certificate that is not included in ca-certificates and which can safely be deleted, because it will just be cached again next time it is encountered. The button will only delete these certificates from the cache, not cause these certificates to be distrusted, as would happen if deleting a root. (Thus, deleting the intermediate certificate is normally almost pointless. Only in combination with this bug does cause trust errors. I think.) Finally, restart Firefox. Firefox will initially display duckduckgo.com with no issues, but only because it is being restored from previous session state cache. Now, refresh the page and you should encounter the certificate error.

One caveat: I don't know how to get my Firefox out of this broken state, and that means I can't test to be certain that following those steps once again will successfully break my Firefox. I just know that it seems to have worked for me the first time I tried it.

Removing all the windows and tabs and then loading duckduckgo.com trigger the error:
duckduckgo.com is most likely a safe site, but a secure connection could not be established. This issue is caused by DigiCert Global Root CA, which is either software on your computer or your network.

And the certificate that triggers the error is always of the type: Software Security Device.

But not all, this works: DigiCert ECC Secure Server CA

New test builds with in-tree nss (firefox-70.0.1-3.nss) are here:
https://koji.fedoraproject.org/koji/packageinfo?packageID=37

(In reply to Martin Stransky from comment #39)
> New test builds with in-tree nss (firefox-70.0.1-3.nss) are here:
> https://koji.fedoraproject.org/koji/packageinfo?packageID=37

The version restore duckduckgo.com (DigiCert SHA2 Secure Server CA) and lenovo.com (DigiCert ECC Secure Server CA).
I will continue testing firefox-70.0.1-3.nss

Using firefox-30.0.1-3.nss.fc30 I had success browsing reddit.com for a while, but suddenly I am getting the "Software is Preventing Firefox From Safely Connecting to This Site" error referencing DigiCert Global Root CA again. It seems it's not a complete fix.

Correction: firefox-70.0.1-3.nss.fc30

It is not fixed with firefox-70.0.1-3.nss -- www.samsung.com(GeoTrust RSA CA 2018):
Firefox does not trust www.samsung.com because its certificate issuer is unknown, the certificate is self-signed, or the server is not sending the correct intermediate certificates.

Added link to Mozilla Bug
https://bugzilla.mozilla.org/show_bug.cgi?id=1593167

I have firefox-70.0-1.fc31.x86_64

What I am observing is that if I open Firefox and imemdiately try to browse to a site like github.com or koji.fedoraproject.org I will get an error. If I wait for few seconds I can usually browse to the site normally.

If instead after opening Firefox I wait several seconds I will not see an error at all.

Hi,

This issue is easily reproducible with following packages in Fedora 30.

[root@localhost ~]# rpm -qa|grep -E '^firefox|^nss|^p11'
firefox-70.0-1.fc30.x86_64
nss-sysinit-3.47.0-2.fc30.x86_64
nss-mdns-0.14.1-3.fc30.x86_64
nss-3.47.0-2.fc30.x86_64
p11-kit-0.23.16.1-1.fc30.x86_64
p11-kit-trust-0.23.16.1-1.fc30.x86_64
nss-util-3.47.0-2.fc30.x86_64
nss-softokn-3.47.0-2.fc30.x86_64
nss-softokn-freebl-3.47.0-2.fc30.x86_64
p11-kit-server-0.23.16.1-1.fc30.x86_64

Am using 'Tab Session Manager' add-on so it instantly open session for few sites with DigiCert CA when run firefox, and all of them are 'prevented' from
opening.

As Jason mention if I wait a little after firefox is started, then start session all sites loads and everything works as expected.

(In reply to Jason Montleon from comment #45)
> I have firefox-70.0-1.fc31.x86_64
> 
> What I am observing is that if I open Firefox and imemdiately try to browse
> to a site like github.com or koji.fedoraproject.org I will get an error. If
> I wait for few seconds I can usually browse to the site normally.
> 
> If instead after opening Firefox I wait several seconds I will not see an
> error at all.

My experience was different, even after waiting a bit (while scrambling to figure out what was going on) they would not load for me.
Downgrading to firefox-69 fixed it for me on my F30 system.

(In reply to Jason Montleon from comment #45)
> I have firefox-70.0-1.fc31.x86_64
> 
> What I am observing is that if I open Firefox and imemdiately try to browse
> to a site like github.com or koji.fedoraproject.org I will get an error. If
> I wait for few seconds I can usually browse to the site normally.
> 
> If instead after opening Firefox I wait several seconds I will not see an
> error at all.

+1, this seems to be happening here too. If I start FF and then wait for the "timer" thing to finish in the gnome-shell top panel (where it says "firefox") and then load duckduckgo.com or netflix.com, everything works just fine. If, however, something tries to open before the timer is finished---a tab from a previous session, for example, the the cert error thing keeps coming up for the rest of the session. So, ff is trying to load something here? How can one get some information on what it is doing maybe?

(In reply to Stefan Dj from comment #46)
> Am using 'Tab Session Manager' add-on so it instantly open session for few
> sites with DigiCert CA when run firefox, and all of them are 'prevented' from
> opening.

Pinning a tab (rightclick -> "Pin Tab") makes it easy to trigger this without an extra addon. Fwiw, when running into this issue https://accounts.firefox.com will show http key pinning issues :)

*** Bug 1773327 has been marked as a duplicate of this bug. ***

stransky, I'm getting this too, but it's erratic enough that I can't easily tell if your test build fixes it.

BTW, this is a better link:

https://koji.fedoraproject.org/koji/buildinfo?buildID=1406664

Nope, not quite fully fixed.

$ rpm -qa firefox
firefox-70.0-2.fc31.x86_64

I removed cert8.db and cert9.db and restarted Firefox, and the first few webpages I tried did not work.  (news.ycombinator.com gave the MITM error and www.wayfair.com gave a different error.)  Then I loaded bugzilla.redhat.com in a different tab in the same browser instance and it worked.  Then I loaded bugzilla.mozilla.org and *that* worked.  Then I reloaded the other two tabs, and they reloaded successfully, too.

Something is wonky with certificate handling,

Same problem/behavior on Fedora 30 with these packages:

     firefox-70.0.1-4.fc30.x86_64
     nss-3.47.0-3.fc30.x86_64
     nss-mdns-0.14.1-3.fc30.x86_64
     nss-pem-1.0.5-1.fc30.x86_64
     nss-softokn-3.47.0-3.fc30.x86_64
     nss-softokn-freebl-3.47.0-3.fc30.i686
     nss-softokn-freebl-3.47.0-3.fc30.x86_64
     nss-sysinit-3.47.0-3.fc30.x86_64
     nss-tools-3.47.0-3.fc30.x86_64
     nss-util-3.47.0-3.fc30.x86_64
     p11-kit-0.23.16.1-1.fc30.x86_64
     p11-kit-server-0.23.16.1-1.fc30.x86_64
     p11-kit-trust-0.23.16.1-1.fc30.x86_64

(In reply to Andy Lutomirski from comment #52)

> Something is wonky with certificate handling,

I have been investigating this for weeks but little progress so far.  What I'm almost sure are:
- This is caused by some behavior difference between the stock libnssckbi.so and p11-kit-trust.so, possibly a race condition, or output order
- When the page is successfully loaded, CKA_NSS_MOZILLA_CA_POLICY is queried, while it isn't in the failed cases (as seen in the log if you run P11_KIT_DEBUG=trust firefox ...)
- The problem can be reproduced even with Firefox 69, but with much lower frequency

For the second point, I started scratch-build of Firefox with debug output; I also tried to build the trunk locally but couldn't reproduce the problem with it.

Is there a Fedora Firefox build that simply disables system certificate store integration entirely?  Being able to reliably use the web seems more important than having system certificates be trusted.

(In reply to Andy Lutomirski from comment #55)
> Is there a Fedora Firefox build that simply disables system certificate
> store integration entirely?  Being able to reliably use the web seems more
> important than having system certificates be trusted.

I fired firefox-70.0.1-5.nss builds for that, will be available at http://koji.fedoraproject.org/koji/packageinfo?packageID=37

There is something strange about the "DigiCert SHA2 Secure Server CA" certificate.  It is supposed to be signed by "DigiCert Global Root CA",
but if you in Firefox look in the Certificate Manager Page and find  the "DigiCert SHA2 Secure Server CA" certificate you wil see in the View->Details page you see the Certificate Hierarchy does not point pack to the signing root certificate.

Could that have something to do with the problem:


Also on that certificate, you can Edit Trust and enable the trust settings.  This seems to make the problem go away for that certificate.

On a clean install of the KDE Plasma spin of Fedora 31, I see this error consistently. Looking at `about:config` on my freshly installed system, I see that security.pki.mitm_canary_issuer is set to "CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US". Both of the user accounts on the system have two firefox profiles. One is "default" and one is "default-release". For some reason, on one of the accounts, prefs.js in the "default-release" profile contains that and the other doesn't.

```
./fhx3n5ez.default-release/prefs.js:user_pref("security.pki.mitm_canary_issuer", "CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US");
```

Clearing that value resolves the issue.

I've got no idea how it could've gotten set in the first place, though, nor why it would be different on one account than the other. I am quite certain that I did not go into about:config and set it myself, though.

It appears the Mozilla folks have made some progress on this.
https://bugzilla.mozilla.org/show_bug.cgi?id=1593167

TLDR: It looks like this is an actual upstream Firefox issue, where Firefox incorrectly caches temporary invalid certificates, and not an NSS issue. The issue seems to just appear more often on Fedora, because NSS certificates load slower and thus the chance is higher that a temporary certificate gets loaded instead of the real one.

(In reply to Jonathan Haas from comment #60)
> TLDR: It looks like this is an actual upstream Firefox issue, where Firefox
> incorrectly caches temporary invalid certificates, and not an NSS issue. The
> issue seems to just appear more often on Fedora, because NSS certificates
> load slower and thus the chance is higher that a temporary certificate gets
> loaded instead of the real one.

How it actually works (if it works correctly):

The website will include its own certificates and the signing certificates as well except the top level CA certificate:

For the bugzilla page, which just showed the proble the certificate chain is:

openssl x509 -noout -subject -issuer -in 1
subject=C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", OU = Information Technology, CN = *.redhat.com
issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA

openssl x509 -noout -subject -issuer -in 2
subject=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA

openssl x509 -noout -subject -issuer -in 3
subject=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA

The files 1, 2, and 3 are the involved certificates exported from Firefox.

"CN = DigiCert SHA2 High Assurance Server CA" comes from the website and is saved in cert9.db (or cert8.db).
"CN = *.redhat.com" also arrives from the website and not stored anywhere.
"CN = DigiCert High Assurance EV Root CA" comes from a local certificate database and it is used to verify "CN = DigiCert SHA2 High Assurance Server CA" which in turn verifies "CN = *.redhat.com".

Some race condition causes the top level certificate not to be found in time when the first verification is done, thus the broken certificate chain, and the web pages therefore marked as "not trusted".

FEDORA-2019-ff27bbf69a has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-ff27bbf69a

Although I have not found bodhi update advisory for F30, there seems to already be F30 builds with same version-release as the one referenced in the F31 advisory:

    https://koji.fedoraproject.org/koji/buildinfo?buildID=1417998

I can reliably reproduce the issue with:

    nss-3.47.0-3.fc30.x86_64
    firefox-70.0.1-4.fc30.x86_64

but it seems to be fixed with nss updated to:

    nss-3.47.1-2.fc30.x86_64

Given it is a bad idea to install something where packager not created advisory yet, I have updated with this command:

    dnf upgrade https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-3.47.1-2.fc30.x86_64.rpm \
                https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-devel-3.47.1-2.fc30.x86_64.rpm \
                https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-softokn-3.47.1-2.fc30.x86_64.rpm \
                https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-softokn-freebl-3.47.1-2.fc30.x86_64.rpm \
                https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-sysinit-3.47.1-2.fc30.x86_64.rpm \
                https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-tools-3.47.1-2.fc30.x86_64.rpm \
                https://kojipkgs.fedoraproject.org//packages/nss/3.47.1/2.fc30/x86_64/nss-util-3.47.1-2.fc30.x86_64.rpm

Thank you!

@Jan Hutař, installing the beta nss packages didn't help and downgrading to Firefox 69.0.3-2 did not solve this problem for me.

@Geoff I have the same setting in my about:config and profile.js. Clearing the value and ensuring it's deleted from profiles.js... does nothing. It gets added back as soon as Firefox is launched.

Something I updated in the past two days on my Fedora 30 system causes this issue. I had already been running Firefox 70 for a while, so I'm flummoxed as to why I can't access Fastmail or DuckDuckGo this morning.

Edit: Both the Firefox 71 beta and Firefox 72 nightly resolve the issue. The DigiCert CA is no longer mentioned as the mitm_canary_issuer in about:config and browsing the DigiCert sites (including Mozilla.org!) works as intended.

(In reply to Eric L. from comment #64)
> @Jan Hutař, installing the beta nss packages didn't help and downgrading to
> Firefox 69.0.3-2 did not solve this problem for me.

Interesting. Could you provide the exact versions of packages you are using, with: rpm -qa firefox nss p11-kit?

Hey, Daiki. I just read your comment on https://bugzilla.mozilla.org/show_bug.cgi?id=1593167#c27 , coincidentally.

Here are my current packages:
```
[root@schwarz eric]# rpm -qa firefox nss p11-kit
p11-kit-0.23.16.1-1.fc30.x86_64
firefox-70.0.1-4.fc30.x86_64
p11-kit-0.23.16.1-1.fc30.i686
nss-3.47.1-2.fc30.i686
nss-3.47.1-2.fc30.x86_64
```

Because Firefox now uses separate profiles for beta and nightly installs, I spent a load of time fiddling with it to try and get the beta install functioning like my normal install. LastPass doesn't work on the beta, for some reason, so I gave up and switched back to 70.0.1-4. I stumbled across a fix on the Manjaro forums https://forum.manjaro.org/t/digicert-invalid-certificate-in-firefox-after-system-upgrade-potential-rogue-certificate/109242/13

If you set the relevant cert to have the "This certificate can identify websites" setting, Firefox 70 can again launch a session with tabs behind Digicert CAs.

nss-3.47.1-2.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-ff27bbf69a

Installed a fresh copy of Fedora 31 on a different laptop. DuckDuckGo and all other DigiCert sites worked on the Firefox 69 + whatever NSS versions shipped with the FC31 XFCE spin. Doing a dnf update brings the same errors as the Firefox 70 + NSS 3.47.1-2 on Fedora 30.

Updated that Fedora 30 to Fedora 31, where the bug obviously persists. The nss-3.47.1-2.fc31 testing update did not fix the problem in FC31.

Mozilla has updated the upstream bug with the following details:

Bug 1593167, certdb: propagate trust information if trust module is loaded afterwards 
When the builtin trust module is loaded after some temp certs being created, these temp certs are usually not accompanied by trust information. This causes a problem in Firefox as it loads the module from a separate thread while accessing the network cache which populates temp certs.

This change makes it properly roll up the trust information, if a temp cert doesn't have trust information.

link: https://phabricator.services.mozilla.com/D54726

----

Bug 1593167, ensure that loadable certs are loaded when TransportSecurityInfo::Read is called 
Previously, there was a race condition: when the first time
CERT_NewTempCertificate is called, the nssckbi module might not be
loaded, and thus the lookup fails. In that case the function creates a
temp certificate without any trust and put it in the cache, which
later interferes with certificate chain validation.

link: https://phabricator.services.mozilla.com/D54135

FEDORA-2019-ff27bbf69a has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-ff27bbf69a

(In reply to Eric L. from comment #69)
> Installed a fresh copy of Fedora 31 on a different laptop. DuckDuckGo and
> all other DigiCert sites worked on the Firefox 69 + whatever NSS versions
> shipped with the FC31 XFCE spin. Doing a dnf update brings the same errors
> as the Firefox 70 + NSS 3.47.1-2 on Fedora 30.
> 
> Updated that Fedora 30 to Fedora 31, where the bug obviously persists. The
> nss-3.47.1-2.fc31 testing update did not fix the problem in FC31.

We've come up with a simpler (and possibly more comprehensive) fix; please try 3.47.1-4 from:
https://bodhi.fedoraproject.org/updates/FEDORA-2019-8fbc65ef9e

If it doesn't work, provide logs by running firefox under: P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5"

nss-3.47.1-4.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-ff27bbf69a

nss-3.47.1-4.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-8fbc65ef9e

The problem persists with the following packages installed:

Installed Packages
firefox.x86_64                                                                   70.0.1-4.fc30                                                    @updates        
nss.i686                                                                         3.47.1-4.fc30                                                    @updates-testing
nss.x86_64                                                                       3.47.1-4.fc30                                                    @updates-testing
nss-mdns.i686                                                                    0.14.1-3.fc30                                                    @fedora         
nss-mdns.x86_64                                                                  0.14.1-3.fc30                                                    @fedora         
nss-pem.x86_64                                                                   1.0.5-1.fc30                                                     @fedora         
nss-softokn.i686                                                                 3.47.1-4.fc30                                                    @updates-testing
nss-softokn.x86_64                                                               3.47.1-4.fc30                                                    @updates-testing
nss-softokn-freebl.i686                                                          3.47.1-4.fc30                                                    @updates-testing
nss-softokn-freebl.x86_64                                                        3.47.1-4.fc30                                                    @updates-testing
nss-sysinit.x86_64                                                               3.47.1-4.fc30                                                    @updates-testing
nss-tools.x86_64                                                                 3.47.1-4.fc30                                                    @updates-testing
nss-util.i686                                                                    3.47.1-4.fc30                                                    @updates-testing
nss-util.x86_64                                                                  3.47.1-4.fc30                                                    @updates-testing

Tested on https://www.mozilla.org/ (Error code: MOZILLA_PKIX_ERROR_MITM_DETECTED) and https://www.17track.net/ (Error code: SEC_ERROR_UNKNOWN_ISSUER)

Created attachment 1642301 [details]
Log with P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5"

The same result with firefox-71.0-8.npgo.fc30.x86_64

(In reply to Cezary Zemis from comment #76)
> Created attachment 1642301 [details]
> Log with P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5"

The log looks quite different from what we had seen; the trust information is correctly queried. Are you able to reproduce it with a fresh new profile (firefox -P) or on a different network?

Cannot reproduce on another account with new profile.

Solved by re-creating cert9.db in existing profile.

I still have the problem on fedora silverblue 31 with a new firefox profile.

nss-3.47.1-4.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

nss-3.47.1-4.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

I'm closing this as I haven't heard any more complaints after 3.47.1-4 builds being pushed to stable.

(In reply to monterr from comment #81)
> I still have the problem on fedora silverblue 31 with a new firefox profile.

I suppose silverblue image didn't contain the updated nss at that time as you mentioned on the upstream bug report:
https://bugzilla.mozilla.org/show_bug.cgi?id=1602760#c3

Created attachment 1649212 [details]
P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5" firefox > firefox.log 2>&1

P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5" firefox > firefox.log 2>&1

(In reply to william.garber from comment #85)
> Created attachment 1649212 [details]
> P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5" firefox > firefox.log 2>&1
> 
> P11_KIT_DEBUG=trust MOZ_LOG="certverifier:5" firefox > firefox.log 2>&1

Why did you attach this? This bug is closed.

I am experiencing the same problem (again) after latest update, it seems random (for example: https://mpi.netcetera.com/). No problem with Chromium.

$ rpm -q firefox nss
firefox-72.0.2-1.fc30.x86_64
nss-3.49.2-1.fc30.x86_64

This is not the same problem, mpi.netcetera.com is serving an incomplete chain: https://www.ssllabs.com/ssltest/analyze.html?d=mpi.netcetera.com&latest -- This is most likely something they should fix on their end.

Thanks for the clarification. But why Chromium has not the same problem? It seems to recognize the whole chain.

After some investigation: https://stackoverflow.com/questions/30360581/certificate-chain-valid-in-chrome-not-firefox-or-android