Bug 2003363 - wget doesn't trust some Microsoft sites
Summary: wget doesn't trust some Microsoft sites
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: gnutls
Version: 37
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Red Hat Crypto Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2024296 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-09-11 14:28 UTC by Zoltan Boszormenyi
Modified: 2023-12-05 21:02 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-12-05 21:02:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gitlab gnutls gnutls issues 1332 0 None None None 2022-03-02 20:48:41 UTC
Red Hat Issue Tracker FC-309 0 None None None 2021-10-18 10:29:31 UTC

Internal Links: 2024296

Description Zoltan Boszormenyi 2021-09-11 14:28:34 UTC
Description of problem:

I use Fedora 33 and 34 on my Yocto build hosts and the resulting OS happens to use Mono and software using Mono.

Yocto's bitbake uses wget to download http and https protocol items from a package recipe's SRC_URI. nuget.exe is one such item but it's download fails:

$ LANG=C wget https://dist.nuget.org/win-x86-commandline/v5.2.0/nuget.exe
--2021-09-11 16:23:31--  https://dist.nuget.org/win-x86-commandline/v5.2.0/nuget.exe
Resolving dist.nuget.org (dist.nuget.org)... 152.199.23.209
Connecting to dist.nuget.org (dist.nuget.org)|152.199.23.209|:443... connected.
ERROR: The certificate of 'dist.nuget.org' is not trusted.

On the other hand, going to dist.nuget.org via a browser poses no issue and the certificate is valid. curl also has no problem with downloading:

$ LANG=C curl --output nuget.exe https://dist.nuget.org/win-x86-commandline/v5.2.0/nuget.exe
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5608k  100 5608k    0     0  16.5M      0 --:--:-- --:--:-- --:--:-- 16.6M

Version-Release number of selected component (if applicable):

wget-1.21.1-3.fc34.x86_64
ca-certificates-2021.2.50-1.0.fc34.noarch
gnutls-3.7.2-1.fc34.x86_64

How reproducible:

Always.

Steps to Reproduce:
1. Use wget to download any version of nuget.exe from https://dist.nuget.org
2.
3.

Actual results:

Download fails because wget doesn't trust the certificate.

Expected results:

Successful download, the same with  curl.

Additional info:

Comment 1 Michal Ruprich 2021-10-18 08:48:25 UTC
Hey Zoltan,

seems like once again this is caused by the fact that in Fedora, wget is built with gnutl and curl with openssl. I tried to rebuild wget with openssl and voila:

# wget  https://dist.nuget.org/win-x86-commandline/v5.11.0/nuget.exe
--2021-10-18 04:46:07--  https://dist.nuget.org/win-x86-commandline/v5.11.0/nuget.exe
Resolving dist.nuget.org (dist.nuget.org)... 152.199.4.184
Connecting to dist.nuget.org (dist.nuget.org)|152.199.4.184|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6786464 (6.5M) [application/x-msdownload]
Saving to: ‘nuget.exe’

nuget.exe                     100%[================================================>]   6.47M  --.-KB/s    in 0.1s    

2021-10-18 04:46:08 (65.6 MB/s) - ‘nuget.exe’ saved [6786464/6786464]

Comment 2 Zoltan Boszormenyi 2021-10-18 09:28:48 UTC
I suspected that.
Will you or someone else fix gnutls in Fedora to actually use a ca-certificates from the system, or will gnutls be retired gradually from Fedora?

Comment 3 Michal Ruprich 2021-10-18 10:27:53 UTC
I am going to switch this to gnutls so that Daiki can take a look whether this is solvable. In the end, I have no problem with rebuilding wget in Fedora with openssl instead.

gnutls-cli -d 1  dist.nuget.org -p 443
Processed 146 CA certificate(s).
Resolving 'dist.nuget.org:443'...
Connecting to '152.199.4.184:443'...
- Certificate type: X.509
- Got a certificate list of 2 certificates.
- Certificate[0] info:
 - subject `CN=*.nuget.org,O=Microsoft Corporation,L=Redmond,ST=WA,C=US', issuer `CN=Microsoft Azure TLS Issuing CA 05,O=Microsoft Corporation,C=US', serial 0x330017f392df3169646870385900000017f392, RSA key 2048 bits, signed using RSA-SHA384, activated `2021-08-03 22:49:43 UTC', expires `2022-07-29 22:49:43 UTC', pin-sha256="7gkSvGqS4XDwl3gp0t29UI4+DhjOIkr/NU86obw0bU4="
	Public Key ID:
		sha1:1c54e6eb8d5d83fff91a98314a8430b578b38924
		sha256:ee0912bc6a92e170f0977829d2ddbd508e3e0e18ce224aff354f3aa1bc346d4e
	Public Key PIN:
		pin-sha256:7gkSvGqS4XDwl3gp0t29UI4+DhjOIkr/NU86obw0bU4=

- Certificate[1] info:
 - subject `CN=Microsoft Azure TLS Issuing CA 05,O=Microsoft Corporation,C=US', issuer `CN=DigiCert Global Root G2,OU=www.digicert.com,O=DigiCert Inc,C=US', serial 0x0d7bede97d8209967a52631b8bdd18bd, RSA key 4096 bits, signed using RSA-SHA384, activated `2020-07-29 12:30:00 UTC', expires `2024-06-27 23:59:59 UTC', pin-sha256="4i4h0jN9NROr1xKJI+TQ1Q/ZIfUjPMXtmWUsDR3Pjiw="
- Status: The certificate is NOT trusted. The received OCSP status response is invalid. 
*** PKI verification of server certificate failed...
*** Fatal error: Error in the certificate.

The dates seem fine to me...

Comment 4 Anderson Sasaki 2021-10-18 12:21:41 UTC
Sorry for intruding here, I suspect the issue is not on gnutls, but on the algorithm used by the OCSP responder (RSA-SHA1). The key line is:

- Status: The certificate is NOT trusted. The received OCSP status response is invalid.

When I try this with DEFAULT policy in crypto-policies the certificate verification fails:

$ gnutls-cli --save-ocsp ocsp.pem --save-cert chain.pem --port=443 dist.nuget.org

Then running:

$ ocsptool --ask --load-chain chain.pem

Tells me:

Verifying OCSP Response: One of the involved algorithms has insufficient security level.

When I switch to LEGACY policy in crypto-policies and try to connect, it is successful:

$ sudo update-crypto-policies --set LEGACY
$ gnutls-cli --save-ocsp ocsp.pem --save-cert chain.pem --port=443 dist.nuget.org

You can see on the ocsptool output that the OCSP responder used RSA-SHA1 to sign the response.

Comment 5 Michal Ruprich 2021-10-18 12:45:08 UTC
Good point, I was trying to figure out why the OCSP is not working but was not aware of the ocsptool. So the question is, why is curl(or wget with openssl) allowing this OCSP response to work even when the crypto-policy is set to DEFAULT? Seems like the definition of the DEFAULT states this:

"Signature algorithms: with SHA-256 hash or better (not DSA)" 

and that should be applied since F33. So gnutls in fact works correctly but openssl ignores that?

Comment 6 Anderson Sasaki 2021-10-18 15:18:15 UTC
Yes, it seems to be the case: gnutls is correctly following crypto-policies while openssl is ignoring it.

Maybe this is a bug in openssl, specifically on OCSP signature verification, but further investigation is required to make it clear.

@Sahana: Could you take a look on this?

Comment 7 Tom Deseyn 2021-11-24 20:19:28 UTC
@rzio (Microsoft) said in https://github.com/dotnet/core/issues/6830#issuecomment-954304508:

```
According to RFC 6960 , (specifically section 4.3) OCSP responders can return responses signed using RSA with SHA1. RFC 5019 which standardizes "The Lightweight Online Certificate Status Protocol (OCSP) Profile for High-Volume Environments" states that only SHA1 should be supported.
Let's Enrypt had a similar issue filed and reading through it's resolution it seems to me that gnu_tls should support SHA1 for OCSP response.

OneOCSP is compliant to the RFCs similarly to the LE OCSP responder.
```

Comment 8 Daiki Ueno 2021-11-25 13:19:51 UTC
The latter claim does is not applicable here: RFC 5019 only mandates hash algorithms for CertID.issuerNameHash and CertID.issuerKeyHash, and does not say anything about signature algorithms on OSCP response.

Side note: Firefox (NSS) in Fedora also accepts SHA-1 signatures on OCSP responses, but I would say it's accidentally allowed as a side effect of bug 1908018.

Comment 9 Michael Catanzaro 2021-12-09 21:07:43 UTC
(In reply to Anderson Sasaki from comment #6) 
> Maybe this is a bug in openssl, specifically on OCSP signature verification,
> but further investigation is required to make it clear.

How are you testing this? Are you sure you actually enabled OCSP signature verification? Doing this with OpenSSL is not straightforward, whereas gnutls-cli checks stapled OCSP responses by default.

Comment 10 Michael Catanzaro 2021-12-09 21:08:05 UTC
*** Bug 2024296 has been marked as a duplicate of this bug. ***

Comment 11 Michael Catanzaro 2021-12-09 21:41:26 UTC
(In reply to Michael Catanzaro from comment #9)
> How are you testing this? Are you sure you actually enabled OCSP signature
> verification? Doing this with OpenSSL is not straightforward, whereas
> gnutls-cli checks stapled OCSP responses by default.

Ah, sorry, didn't read enough. I see you recompiled wget to use OpenSSL rather than GnuTLS. I suspect wget simply does not enable verification of stapled OCSP responses. glib-networking uses SSL_get_tlsext_status_ocsp_resp() to do this, but there is a warning in the code about how we're not checking Must-Staple and therefore it offers no security benefits. In contrast, GnuTLS will handle OCSP stapling automatically as long as you use the right certificate verification API, much easier.

Comment 12 Michael Catanzaro 2022-02-21 14:08:00 UTC
Ping, we really need to decide what to do about this. I know it's a tough problem, but we shouldn't leave Fedora broken in the meantime. I suggest loosening the crypto policy for OCSP (and only for OCSP). In the meantime, my workaround for Fedora 35 is to disable certificate revocation checks altogether (sabotage-revocation-checks.patch in the glib-networking package).

Comment 13 Alexander Sosedkin 2022-03-03 09:40:52 UTC
To briefly summarize the relevant parts of the recent discussion:

1. Relaxing the policy to just accept SHA-1 signed stapled responses in OCSP is a no-go considering RHEL-9 lifetime.
2. GnuTLS should at least disambiguate the "weak signature" cause of failure to give applications a chance to recover.

Comment 14 Ben Cotton 2022-05-12 16:51:31 UTC
This message is a reminder that Fedora Linux 34 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '34'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 34 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 15 Michael Catanzaro 2022-07-26 13:24:40 UTC
Certificate revocation support remains disabled in glib-networking. I was planning to keep it disabled until this bug is fixed. But I'm not sure if it's really still needed at all:

 * dist.nuget.org seems to be fixed
 * So does source.redhat.com
 * And also account.live.com

Those are the only three websites that were known to be broken afaik, so... maybe we're good now? It looks like both Microsoft and DigiCert are no longer signing stapled OCSP responses with SHA-1?

It seems they even changed the Baseline Requirements to prohibit this:

https://cabforum.org/2022/01/26/ballot-sc53-sunset-for-sha-1-ocsp-signing/

So I think we're good now. I'm going to go ahead and reenable certificate revocation checking in rawhide. I suspect this bug can be closed.

Comment 16 Ben Cotton 2022-08-09 13:37:52 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 37 development cycle.
Changing version to 37.

Comment 17 Aoife Moloney 2023-11-23 00:06:18 UTC
This message is a reminder that Fedora Linux 37 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '37'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 37 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 18 Aoife Moloney 2023-12-05 21:02:03 UTC
Fedora Linux 37 entered end-of-life (EOL) status on None.

Fedora Linux 37 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.