RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1394948 - nodejs core dumps when https used with startssl server certificate
Summary: nodejs core dumps when https used with startssl server certificate
Keywords:
Status: CLOSED DUPLICATE of bug 1276310
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openssl
Version: 7.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Tomas Mraz
QA Contact: BaseOS QE Security Team
URL:
Whiteboard:
: 1402372 1413370 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-14 20:33 UTC by Bobby Krupczak
Modified: 2017-04-03 14:34 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-03 14:34:01 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
test.js to generate node core dump (721 bytes, text/plain)
2016-11-14 20:33 UTC, Bobby Krupczak 		no flags Details
Backtrace of crash (19.31 KB, text/plain)
2016-11-15 14:54 UTC, Stephen Gallagher 		no flags Details
View All

Description Bobby Krupczak 2016-11-14 20:33:26 UTC 
Created attachment 1220531 [details]
test.js to generate node core dump

Description of problem:

simple nodejs https server core dumps when inbound https connection attempted.  server cert is from startssl.com

Version-Release number of selected component (if applicable):

epel-release-7-8.noarch
nodejs-6.9.1-1.el7.x86_64
openssl-libs-1.0.1e-60.el7.x86_64

How reproducible:

Attached is simple test.js server code that creates simple https server.

Steps to Reproduce:
1. use startssl server cert (I can provide if you'd like)
2. run /bin/node test.js 
3. core dump

Actual results:

node core dumps when incoming https request

Expected results:

node should not core dump

Additional info:

I placed traces, server certs, test.js at github because I originally thought the bug was with node.  However, on my ec2 instance, when I downloaded/installed nodejs build/binaries, the code does not core dump.

https://github.com/nodejs/node/issues/9551
Comment 1 Stephen Gallagher 2016-11-14 20:47:24 UTC 
I'm looking into this. So far I cannot reproduce the issue locally, but I'm attempting to set up a StartSSL account so I can see if the problem is unique to certificates issued through them.
Comment 2 Bobby Krupczak 2016-11-14 20:50:11 UTC 
I'm happy to give you my certs to use or I can give you access to my ec2 instance.

Thanks,

Bobby
Comment 3 Bobby Krupczak 2016-11-14 21:23:08 UTC 
Oh yeah, one more item on the certificate.  I take the cert given to me by ssl and I append the startssl intermediate cert to the end of the cert pem file.
Comment 4 Stephen Gallagher 2016-11-14 21:32:54 UTC 
OK, I just did the following:

* Created a CSR for one of my domains
* Got a certificate issued for that CSR from StartSSL
* Installed CentOS 7.2
* Updated to openssl-libs-1.0.1e-51.el7_2.7.x86_64.rpm
* Installed epel-release
* Installed nodejs
* Ran your test script
* Ran `openssl s_client -connect <hostname>:8443 -CAfile 1_root_bundle.crt`
* Updated to openssl-libs-1.0.1e-60.el7.x86_64
* Ran your test script
* Ran `openssl s_client -connect <hostname>:8443 -CAfile 1_root_bundle.crt`

Both attempts at `openssl s_client` succeeded and did not trigger a segmentation fault in Node.js.

I can only assume that there is something unusual with your specific environment. Do you perhaps have unusual characters in your CN or subjectAltNames for the certificate?
Comment 5 Stephen Gallagher 2016-11-14 21:35:53 UTC 
(In reply to Bobby Krupczak from comment #3)
> Oh yeah, one more item on the certificate.  I take the cert given to me by
> ssl and I append the startssl intermediate cert to the end of the cert pem
> file.

Why do you do that? 

OK, I can confirm *that* will crash the application, but what is the purpose of putting the intermediate certificate in with the service certificate?
Comment 6 Tom Hughes 2016-11-14 21:57:51 UTC 
Presumably because applications using openssl will often read it from there...

Specifically if they use SSL_CTX_use_certificate_chain_file to load the certificate then they will accept an intermediate certificate that follows it.

No idea which routine Node uses to load the certificate though.
Comment 7 Bobby Krupczak 2016-11-14 23:00:56 UTC 
Hey!

One thing to mention.  You tested on centos 7.2 and my ec2 instance is running RHEL 7.3 (centos 7.3).

Re Stephen's confirmation that the intermediate cert will crash node . . . I guess its still a node bug?  Or, openssl bug?  Even if I feed an erroneous cert, the code should not crash.

I added the intermediate cert in the server cert file because my understanding was that one needed it so that the server code would think the cert is legit.  I probably mis-understood the cert install instructions.

I just tested the epel node build with my cert with the intermediate cert removed and it did not crash.  You are correct.  That fixed that problem.

Should I close this ticket?

I will also update the github issue with this revelation.

Bobby
Comment 8 Stephen Gallagher 2016-11-15 14:54:08 UTC 
Created attachment 1220859 [details]
Backtrace of crash

Attaching a backtrace produced while running openssl-libs-1.0.1e-51.el7_2.7.x86_64

I used the same certificates I generated in comment #4, except that I appended the intermediate certificate to the service certificate. I modifed the test.js slightly, just to look for the combined cert.

I can confirm that the issue only happens on the RHEL 7.2 and 7.3 OpenSSL packages. I actually did an EPEL 7 scratch-build of the latest Fedora OpenSSL package (http://koji.fedoraproject.org/koji/taskinfo?taskID=16463661) and tried that; it did not crash.
Comment 9 Stephen Gallagher 2016-11-15 14:55:56 UTC 
Reassigning the issue to OpenSSL, as the problem is definitely on that end.
Comment 11 Tomas Mraz 2016-11-16 13:25:23 UTC 
I need to know what is the method that nodeJS calls when loading the certificate.
Comment 12 Tom Hughes 2016-11-16 13:38:55 UTC 
It's a bit involved, but think it is probably SecureContext::SetCert (https://github.com/nodejs/node/blob/master/src/node_crypto.cc#L633) that winds up doing it, which is using SSL_CTX_use_certificate_chain.

Of course if the reporter can get a backtrace that will help confirm...
Comment 13 Stephen Gallagher 2016-11-16 14:01:53 UTC 
(In reply to Tom Hughes from comment #12)
> It's a bit involved, but think it is probably SecureContext::SetCert
> (https://github.com/nodejs/node/blob/master/src/node_crypto.cc#L633) that
> winds up doing it, which is using SSL_CTX_use_certificate_chain.
> 
> Of course if the reporter can get a backtrace that will help confirm...

A backtrace doesn't help (and I attached mine to the ticket). The problem is that the failure doesn't happen at load-time, it happens the first time the loaded value is used after that.

Tomas, I don't really know the internals of Node.js, but Tom Hughes does. I'll trust his interpretation and clear NEEDINFO.
Comment 14 Tomas Mraz 2016-11-16 17:21:34 UTC 
It would be also interesting to see if the crash happens with any other (valid) server+intermediate certificates.
Comment 15 Chris Northwood 2016-12-07 13:20:44 UTC 
*** Bug 1402372 has been marked as a duplicate of this bug. ***
Comment 16 Stephen Gallagher 2017-01-16 13:00:32 UTC 
*** Bug 1413370 has been marked as a duplicate of this bug. ***
Comment 17 Victor 2017-03-24 17:14:15 UTC 
I'm getting the same problem with CentOS 7.2.1511, EPEL, and OpenSSL 1.0.1e-fips. We have a Thawte cert. Removing the intermediate certificates fixes the segfault issue. 

However, without the intermediate certificates we are getting an error on Chrome (Android) with our secure web sockets (wss): "No certificates found The app Chrome has requested a certificate. Choosing a certificate will let the app use this identity with servers now and in the future...". My guess is this error is because we have removed the intermediate certificates. Strangely, this error only occurs on Chrome in Android. iOS and all desktop browsers I've tested seem fine.
Comment 18 Victor 2017-03-27 19:57:34 UTC 
I found out that I'm getting the segfault issue with the EPEL version of Node.js 6.9.4. When I downgrade to version 4.8.1 provided by NodeSource I no longer get any segfaults when using an intermediate certificate. Great success!
Comment 19 Tomas Mraz 2017-04-03 14:34:01 UTC 
It would still be useful to find why the crash with 1.0.1e happens but apparently this will be fixed by the rebase to 1.0.2.

*** This bug has been marked as a duplicate of bug 1276310 ***
Note You need to log in before you can comment on or make changes to this bug.