Bug 1657331

Summary: curl fails on Slack repo with SSL error
Product: [Fedora] Fedora Reporter: Chris Adams <linux>
Component: opensslAssignee: Tomas Mraz <tmraz>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 29CC: john.j5live, jorton, kdudka, linux, paul, tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-19 16:27:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chris Adams 2018-12-07 16:43:17 UTC
I am trying to install the Linux Slack client on Fedora 29 from their repository. I used the Slack repo from COPR with:

   dnf copr enable jdoss/slack-repo
   dnf install slack-repo
   dnf install slack

dnf just says it failed to sync the Slack repo metadata. When I try to fetch the repomd.xml with curl, I get:

$ curl -LO https://packagecloud.io/slacktechnologies/slack/fedora/21/x86_64/repodata/repomd.xml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to packagecloud-repositories.s3.dualstack.us-west-1.amazonaws.com:443 

Every other tool I use (wget, Firefox, Chrome) fetches the file okay (but of course, dnf uses libcurl, so this keeps dnf from seeing that repo).

I can't find any issue with the site (the original URL gets a long redirect to AWS) - I've run it through an SSL tester and it looks okay. Seems like an issue with curl?

Comment 1 Kamil Dudka 2018-12-10 13:26:54 UTC
I have seen this before, e.g.:

https://github.com/curl/curl/issues/2294

... but I was not able to reproduce it locally.  Could you please try it with --ciphers DEFAULT ?

Comment 2 Chris Adams 2018-12-10 13:54:50 UTC
I still get an error with --ciphers DEFAULT. One difference though is that I don't even get redirected to the AWS host with --ciphers DEFAULT; it fails at packagecloud.io. This works from another Fedora 29 system that is also on a different network (packagecloud.io resolves to the same IP in both places).

Comment 3 Kamil Dudka 2018-12-10 14:07:12 UTC
Another issue (linked from the above one) describes a situation where the target was blocked by the Internet provider:

https://github.com/rust-lang/rustup.rs/issues/1404

Are you able to connect the host(s) with s_client?

$ openssl s_client -port 443 -host ...

Comment 4 Chris Adams 2018-12-10 14:26:48 UTC
So on the failing system, I am the Internet provider; I'm pretty sure it's not that. :)

Poked some more (all from the failing system):

- curl to the packagecloud.io URL: successfully connects, gets redirected to an AWS host, that connection fails with SSL_ERROR_SYSCALL
- curl --ciphers DEFAULT: connection to packagecloud.io fails with SSL_ERROR_SYSCALL
- openssl s_client to packagecloud.io: successful connection, gets the redirect
- openssl s_client to the AWS host: connection fails (gets the cert but has a write error and closes the connection)
- openssl s_client to the AWS host with -cipher ECDHE-RSA-AES256-SHA (just kind of picked a cipher at random): works, gets the file

So maybe there's a TLS issue... but why is there a difference between "curl" and "curl --ciphers DEFAULT"?

Comment 5 Kamil Dudka 2018-12-10 14:49:45 UTC
curl respects the system-wide crypto policy by default (see bug #1483972).  You should be able to change the policy by editing /etc/crypto-policies/config and running update-crypto-policies afterwards.

Comment 6 Kamil Dudka 2018-12-19 13:34:42 UTC
Are you able to connect with `curl --ciphers ECDHE-RSA-AES256-SHA ...` ?

Comment 7 Chris Adams 2018-12-19 14:15:17 UTC
Yes, that works.

It looks like openssl is choosing ECDHE-RSA-AES128-GCM-SHA256 by default, and that's when the connection fails. I can successfully use that cipher with "openssl s_client" on another system (CentOS 7), so it would seem there's an issue with openssl on Fedora 29. Since this appears to be an openssl issue, not curl, I'm moving this bug to openssl.

Comment 8 Tomas Mraz 2018-12-19 15:28:30 UTC
Does openssl s_client -connect packagecloud-repositories.s3.dualstack.us-west-1.amazonaws.com:443

work for you on F29 or not?

The problem is I cannot reproduce the issue here with openssl-1.1.1-3.fc29 neither with curl nor with openssl. And the cipher used on s_client is ECDHE-RSA-AES128-GCM-SHA256.

Comment 9 Chris Adams 2018-12-19 16:02:08 UTC
It depends on where I try from - it looks like packagecloud-repositories.s3.dualstack.us-west-1.amazonaws.com resolves to different IPs from different places. When I try from the system with a problem, it resolves to 52.219.116.9, and that's the IP that I can't connect to.

However, it doesn't appear that it's just the cipher - I just tried switching system policy between DEFAULT and LEGACY - with LEGACY, s_client can connect and work, but with DEFAULT it can't... but both appear to be using the same cipher (ECDHE-RSA-AES128-GCM-SHA256). Is there more debugging I can do? I haven't had to debug an SSL issue like this, so not sure what else to try.

Comment 10 Chris Adams 2018-12-19 16:27:01 UTC
Ugh, this appears to be related to a port-mirrored content filter on my office network (that I forgot about because it's not in-line)... it apparently doesn't like something about the TLS communication in DEFAULT-policy and resets the TCP session, but it ignores the LEGACY-policy connection (so it works). It's weird that it only seems to be a problem for this one site, but whatever.

I apologize for wasting everybody's time with this.