Bug 1241172

Summary: Certificate verification fails with multiple https urls [el7/nss]
Product: Red Hat Enterprise Linux 7 Reporter: rhbug
Component: nssAssignee: nss-nspr-maint <nss-nspr-maint>
Status: CLOSED ERRATA QA Contact: Hubert Kario <hkario>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.1CC: alexander.naumann, apmukher, aurelien, ben.r.xiao, bugzilla, d.bz-redhat, desintegr, d.fedora, dossow, emaldona, FlorianFranzen, fredrik, g.d0b3rm4n, hkario, hlx98007, igeorgex, james.hogarth, jbnance, karli.sjoberg, kdudka, kengert, marek, mdsreg_rhbz, me, michal.bruncko, mpoole, mtolson, nkinder, redhatbugs, red-hat-bugzilla, redhat-bugzilla, redhat, robert.scheck, ville, wvoyek, xrobau
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: nss-3.21.0-13.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1260678 (view as bug list) Environment:
Last Closed: 2016-11-04 03:55:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1104597    
Bug Blocks: 1260678, 1269660    

Description rhbug 2015-07-08 16:09:56 UTC
Description of problem:

Programs using curl or libcurl return HTTP 400 error on subsequent https calls because certificate verification fails. If only one url is requested at a time, verification succeeds.


Version-Release number of selected component (if applicable):

  curl-7.29.0-19.el7.x86_64
  php-5.4.16-36.el7_1.x86_64
  nss-3.19.1-3.el7_1.x86_64


How reproducible:

[user@host htdocs]# curl -v -I https://owncloud.org/ https://www.owncloud.org/
  * About to connect() to owncloud.org port 443 (#0)
  *   Trying 50.30.33.236...
  * Connected to owncloud.org (50.30.33.236) port 443 (#0)
  * Initializing NSS with certpath: sql:/etc/pki/nssdb
  *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
    CApath: none
  * SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
  * Server certificate:
  * 	subject: CN=www.owncloud.org,OU=Domain Control Validated
  * 	start date: Feb 09 23:47:38 2015 GMT
  * 	expire date: Dec 10 10:32:11 2015 GMT
  * 	common name: www.owncloud.org
  * 	issuer: CN=Go Daddy Secure Certificate Authority - G2,OU=http://certs.godaddy.com/repository/,O="GoDaddy.com, Inc.",L=Scottsdale,ST=Arizona,C=US
  > HEAD / HTTP/1.1
  > User-Agent: curl/7.29.0
  > Host: owncloud.org
  > Accept: */*
  > 
  < HTTP/1.1 200 OK
  [...]
  
  * Connection #0 to host owncloud.org left intact
  * About to connect() to www.owncloud.org port 443 (#1)
  *   Trying 50.30.33.236...
  * Connected to www.owncloud.org (50.30.33.236) port 443 (#1)
  *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
    CApath: none
  * SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
  * Server certificate:
  * 	subject: CN=www.owncloud.org,OU=Domain Control Validated
  * 	start date: Feb 09 23:47:38 2015 GMT
  * 	expire date: Dec 10 10:32:11 2015 GMT
  * 	common name: www.owncloud.org
  * 	issuer: CN=Go Daddy Secure Certificate Authority - G2,OU=http://certs.godaddy.com/repository/,O="GoDaddy.com, Inc.",L=Scottsdale,ST=Arizona,C=US
  > HEAD / HTTP/1.1
  > User-Agent: curl/7.29.0
  > Host: www.owncloud.org
  > Accept: */*
  > 
  < HTTP/1.1 400 Bad Request
  [...]
  * Closing connection 1

The only difference I can see is that
  Initializing NSS with certpath: sql:/etc/pki/nssdb
is present on the first call but not on the second. CAfile is mentioned on both calls.

Steps to Reproduce:
1. curl -v -I https://owncloud.org/ https://www.owncloud.org/
2.
3.

Actual results:
HTTP/1.1 200 OK
HTTP/1.1 400 Bad Request


Expected results:
HTTP/1.1 200 OK (for https://owncloud.org/)
HTTP/1.1 301 Moved Permanently (for https://www.owncloud.org/)


Additional info:

This also happens with php 5.6.10 and curl-7.43, both compiled from Fedora src rpm for RHEL7 but only when --with-nss (default) is used. Compiled with --with-ssl (openssl) curl/libcurl/php works as expected.

This prevents e.g. owncloud from working properly (apps / external data cannot be loaded)

Comment 1 Kamil Dudka 2015-07-08 16:33:22 UTC
This looks like bug #1104597 (nss bug), will have a look tomorrow.

Comment 3 Benjamin Xiao 2015-07-08 18:01:45 UTC
Also seeing this on Fedora 22 with up-to-date standard php and curl packages.

Comment 4 Kamil Dudka 2015-07-09 08:50:19 UTC
(In reply to Kamil Dudka from comment #1)
> This looks like bug #1104597 (nss bug), will have a look tomorrow.

Confirmed.  Attachment #902122 [details] fixes the bug.  Flipping the component to nss.

Comment 5 Stefan Neufeind 2015-07-25 20:18:33 UTC
I've heard rumours this also affacts RHEL6, so we might require a backport as well then.

Comment 7 Kamil Dudka 2015-09-03 09:02:58 UTC
A patch for this bug (attachment #902122 [details]) was proposed more than a year ago.  Could you please have a look at that?

Comment 8 Kamil Dudka 2015-09-18 16:04:38 UTC
There is a workaround for curl suggested by Mozilla upstream:

https://github.com/bagder/curl/commit/958d2ffb

Comment 9 Benjamin Xiao 2015-09-22 01:47:20 UTC
New incoming curl package in testing repo seems to fix this issue.

Comment 11 Karli Sjöberg 2015-12-03 20:04:33 UTC
(In reply to Stefan Neufeind from comment #5)
> I've heard rumours this also affacts RHEL6, so we might require a backport
> as well then.

I can confirm this for CentOS 6.7. Any ETA on backport?

Best Regards
Karli Sjöberg

Comment 12 xrobau 2016-01-21 04:52:24 UTC
Confirmed still an issue in the latest 7, but this ticket appears to say that it's fixed.

nss-3.19.1-19.el7_2.x86_64

Comment 13 Kamil Dudka 2016-01-21 07:24:04 UTC
(In reply to xrobau from comment #12)
> but this ticket appears to say that it's fixed.

Nobody says this bug has been fixed in RHEL-7.

Comment 14 Kai Engert (:kaie) (inactive account) 2016-01-22 18:20:35 UTC
This will be fixed upstream, probably in NSS 3.23

Elio, could you please pick up the upstream patch, and carry it downstream, for the next possible NSS build for RHEL 7.3?

Comment 17 James Hogarth 2016-04-04 13:03:28 UTC
Hi is there any information on progress fixing this issue in RHEL?

I'm currently in the middle of packaging the owncloud 8.1.6 update for EPEL7 and this bug is causing issues with the owncloud appstore usage in testing.

The issue at discussion in the oC community is: 

https://github.com/owncloud/core/issues/16255

Investigation lead to this PHP bug:

https://bugs.php.net/bug.php?id=67639

Which then of course lead to identifying this as the NSS bug root causing it.

Comment 18 Kamil Dudka 2016-04-04 13:22:42 UTC
(In reply to James Hogarth from comment #17)
> Investigation lead to this PHP bug:
> 
> https://bugs.php.net/bug.php?id=67639
> 
> Which then of course lead to identifying this as the NSS bug root causing it.

The above example uses libcurl, which is going to have a workaround for this bug of NSS.  See bug #1269855, which is already in ON_QA.  I will not speak for this NSS bug itself though...

Comment 19 James Hogarth 2016-04-04 13:46:17 UTC
Thanks for the pointer to that Kamil

Are you aware if this is due to be an update within EL7.2 or will need to wait for the EL7.3 milestone?

I'd be extremely grateful if you could verify on your test rig the code at the php bug, which uses libcurl of course so should behave with the curl patch discussed.

Owncloud uses guzzlehttp for its requests which ultimately uses the phpcurl libraries so in principal should be fixed by that without needing fix correct the underlying NSS issue.

Comment 20 Kamil Dudka 2016-04-04 14:12:23 UTC
(In reply to James Hogarth from comment #19)
> Are you aware if this is due to be an update within EL7.2 or will need to
> wait for the EL7.3 milestone?

I am not aware of any fix going out sooner than RHEL-7.3.  Feel free to escalate the issue via Product Support if the fix is important for your business.

> I'd be extremely grateful if you could verify on your test rig the code at
> the php bug, which uses libcurl of course so should behave with the curl
> patch discussed.

You can try the patch that was applied on upstream (lib)curl:

    https://github.com/curl/curl/commit/958d2ffb

There is also a Fedora sibling of the bug that contains more info on the topic:

    bug #1104597

Comment 21 Robert Scheck 2016-05-19 18:00:45 UTC
Cross-filed case 01637758 on the Red Hat customer portal to get this moved
on, given there was no visible progress (to us as a customer) for nearly one
year now.

Comment 23 xrobau 2016-05-20 07:36:49 UTC
Concur, it's a bug, it should be fixed, and it's been ignored for 7.1 and 7.2 8-\

Comment 24 Hubert Kario 2016-05-20 09:42:48 UTC
It's my understanding that the issue will be fixed on curl side, see bug 1269855. The release of it is scheduled for 7.3.0.

The curl-side-fix was released for RHEL-6 in the 6.8.0 release, see bug 1269660.

Comment 25 Kamil Dudka 2016-05-20 10:33:41 UTC
Yes, the curl fix is ready to be released in RHEL-7.3.  But please do not use it as an excuse for not releasing or postponing the NSS fix.  Applications using NSS for TLS directly will be affected by this bug even after the update of curl.

The NSS fix is a one-line patch (attachment #902122 [details]), submitted for review 2 years ago, with no known downsides so far.  I see no valid reason for not including the fix in the RHEL-7.3 update of NSS.

Comment 30 errata-xmlrpc 2016-11-04 03:55:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2335.html