Hide Forgot
Description of problem: curl crashes on https:// URLs: $ curl -v https://redhat.com/ * About to connect() to redhat.com port 443 (#0) * Trying 10.4.164.55... * Connected to redhat.com (10.4.164.55) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none Segmentation fault (core dumped) The same document can be retrieved using wget. The backtrace points to ssl3_InitHandshakeHashes(): $ gdb --args curl https://copr-be.cloud.fedoraproject.org/results/lmr/avocado/epel-7-x86_64/ GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "aarch64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/curl...Reading symbols from /usr/bin/curl...(no debugging symbols found)...done. (no debugging symbols found)...done. Missing separate debuginfos, use: debuginfo-install curl-7.29.0-25.el7.aarch64 (gdb) r Starting program: /usr/bin/curl https://copr-be.cloud.fedoraproject.org/results/lmr/avocado/epel-7-x86_64/ [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [New Thread 0x3ffb090f100 (LWP 10673)] [Thread 0x3ffb090f100 (LWP 10673) exited] Program received signal SIGSEGV, Segmentation fault. 0x000003ffb7ecd768 in ssl3_InitHandshakeHashes () from /lib64/libssl3.so (gdb) bt #0 0x000003ffb7ecd768 in ssl3_InitHandshakeHashes () from /lib64/libssl3.so #1 0x000003ffb7ed3330 in ssl3_HandleServerHello () from /lib64/libssl3.so #2 0x000003ffb7ed6910 in ssl3_HandleHandshakeMessage () from /lib64/libssl3.so #3 0x000003ffb7ed8e38 in ssl3_HandleRecord () from /lib64/libssl3.so #4 0x000003ffb7eda118 in ssl3_GatherCompleteHandshake () from /lib64/libssl3.so #5 0x000003ffb7ee3274 in SSL_ForceHandshake () from /lib64/libssl3.so #6 0x000003ffb7f5c498 in nss_connect_common () from /lib64/libcurl.so.4 #7 0x000003ffb7f5464c in Curl_ssl_connect_nonblocking () from /lib64/libcurl.so.4 #8 0x000003ffb7f30558 in https_connecting () from /lib64/libcurl.so.4 #9 0x000003ffb7f3f5d0 in Curl_protocol_connecting () from /lib64/libcurl.so.4 #10 0x000003ffb7f4e548 in multi_runsingle () from /lib64/libcurl.so.4 #11 0x000003ffb7f4ef7c in curl_multi_perform () from /lib64/libcurl.so.4 #12 0x000003ffb7f46d84 in curl_easy_perform () from /lib64/libcurl.so.4 #13 0x0000000000408f54 in operate () #14 0x000000000040288c in main () (gdb) On the same system, yum crashes with a comparable backtrace when any of the repositories has an https:// baseurl. yum was working fine with openssl-1.0.1e-51.el7_2.2.aarch6 and I assume curl was as well. Version-Release number of selected component (if applicable): openssl-1.0.1e-51.el7_2.4.aarch64 curl-7.29.0-25.el7.aarch64 python-pycurl-7.19.0-17.el7.aarch64 yum-3.4.3-132.el7.noarch How reproducible: Always. Steps to Reproduce: See above. Actual results: curl crashes :( Expected results: curl does not crash :) Additional info: None.
(In reply to Andrea Bolognani from comment #0) > yum was working fine with > > openssl-1.0.1e-51.el7_2.2.aarch6 > > and I assume curl was as well. I see no openssl symbols in the above backtrace. This is more likely related to an update of nss* packages.
(In reply to Kamil Dudka from comment #3) > (In reply to Andrea Bolognani from comment #0) > > yum was working fine with > > > > openssl-1.0.1e-51.el7_2.2.aarch6 > > > > and I assume curl was as well. > > I see no openssl symbols in the above backtrace. This is more likely > related to an update of nss* packages. You're right, I assumed /lib64/libssl3.so was from openssl but I see now that it's actually from nss. The current version of that package is nss-3.21.0-1.el7_2.aarch64 while the previous version was nss-3.19.1-19.el7_2.aarch64 Hope this helps.
Thanks! The crash just happened to me on a different aarch64 machine with the same versions of nss* packages installed. It crashes even when using tstclnt, which excludes the possibility of a curl bug: # valgrind /usr/lib64/nss/unsupported-tools/tstclnt -h localhost -p 443 -d sql:/etc/pki/nssdb -V tls1.2:tls1.2 ==12426== Memcheck, a memory error detector ==12426== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==12426== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==12426== Command: /usr/lib64/nss/unsupported-tools/tstclnt -h localhost -p 443 -d sql:/etc/pki/nssdb -V tls1.2:tls1.2 ==12426== ==12426== Invalid read of size 4 ==12426== at 0x48BD768: ssl3_GetPrfHashMechanism (ssl3con.c:3658) ==12426== by 0x48BD768: ssl3_InitHandshakeHashes (ssl3con.c:4153) ==12426== by 0x48C332F: ssl3_HandleServerHello (ssl3con.c:6629) ==12426== by 0x48C690F: ssl3_HandleHandshakeMessage (ssl3con.c:11781) ==12426== by 0x48C8E37: ssl3_HandleHandshake (ssl3con.c:11925) ==12426== by 0x48C8E37: ssl3_HandleRecord (ssl3con.c:12594) ==12426== by 0x48CA117: ssl3_GatherCompleteHandshake (ssl3gthr.c:378) ==12426== by 0x48CACBB: ssl_GatherRecord1stHandshake (sslcon.c:1213) ==12426== by 0x48D2C9F: ssl_Do1stHandshake (sslsecur.c:109) ==12426== by 0x48D3EB3: ssl_SecureRecv (sslsecur.c:1232) ==12426== by 0x48D750F: ssl_Recv (sslsock.c:2397) ==12426== by 0x4B4E9EB: PR_Recv (in /usr/lib64/libnspr4.so) ==12426== by 0x40695F: main (tstclnt.c:1617) ==12426== Address 0x10 is not stack'd, malloc'd or (recently) free'd ==12426== ==12426== ==12426== Process terminating with default action of signal 11 (SIGSEGV) ==12426== Access not within mapped region at address 0x10 ==12426== at 0x48BD768: ssl3_GetPrfHashMechanism (ssl3con.c:3658) ==12426== by 0x48BD768: ssl3_InitHandshakeHashes (ssl3con.c:4153) ==12426== by 0x48C332F: ssl3_HandleServerHello (ssl3con.c:6629) ==12426== by 0x48C690F: ssl3_HandleHandshakeMessage (ssl3con.c:11781) ==12426== by 0x48C8E37: ssl3_HandleHandshake (ssl3con.c:11925) ==12426== by 0x48C8E37: ssl3_HandleRecord (ssl3con.c:12594) ==12426== by 0x48CA117: ssl3_GatherCompleteHandshake (ssl3gthr.c:378) ==12426== by 0x48CACBB: ssl_GatherRecord1stHandshake (sslcon.c:1213) ==12426== by 0x48D2C9F: ssl_Do1stHandshake (sslsecur.c:109) ==12426== by 0x48D3EB3: ssl_SecureRecv (sslsecur.c:1232) ==12426== by 0x48D750F: ssl_Recv (sslsock.c:2397) ==12426== by 0x4B4E9EB: PR_Recv (in /usr/lib64/libnspr4.so) ==12426== by 0x40695F: main (tstclnt.c:1617) ==12426== If you believe this happened as a result of a stack ==12426== overflow in your program's main thread (unlikely but ==12426== possible), you can try to increase the size of the ==12426== main thread stack using the --main-stacksize= flag. ==12426== The main thread stack size used in this run was 8388608. ==12426== ==12426== HEAP SUMMARY: ==12426== in use at exit: 592,028 bytes in 2,202 blocks ==12426== total heap usage: 4,183 allocs, 1,981 frees, 1,523,672 bytes allocated ==12426== ==12426== LEAK SUMMARY: ==12426== definitely lost: 0 bytes in 0 blocks ==12426== indirectly lost: 0 bytes in 0 blocks ==12426== possibly lost: 212,943 bytes in 555 blocks ==12426== still reachable: 379,085 bytes in 1,647 blocks ==12426== suppressed: 0 bytes in 0 blocks ==12426== Rerun with --leak-check=full to see details of leaked memory ==12426== ==12426== For counts of detected and suppressed errors, rerun with: -v ==12426== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) Segmentation fault
*** Bug 1311559 has been marked as a duplicate of this bug. ***
The nss-3.21.0-1.el7_2 build is known to be broken. Unfortunately we cannot yet create a replacement build, because we must keep an older nss-util in the buildroot for an upcoming z-stream errata. Please use an older nss-3.19.* package for now. I believe the broken nss.rpm has already been removed from the most recent buildroot.
the broken nss-3.21 build, and all other related base libraries like nss-util-3.21 and nspr-4.11, have been untagged from the 7.2.z candidate tag. Your next builds should pick up the older pacakges versions, and work again.
I'm closing this bug, because it should have been resolved by the changes in z-stream candidate tags. If you continue to experience this bug when doing a fresh build against the updated tags, please reopen it.
Kai, could you please fill in the Fixed In Version field? Btw. I find it confusing that this bug is already closed. I spent non-trivial amount of time debugging it this morning because I found no open bug describing the crash (in my case reported against curl). This bug should IMO stay open until QE verifies it is fixed.
*** Bug 1312895 has been marked as a duplicate of this bug. ***
(In reply to Kamil Dudka from comment #14) > Btw. I find it confusing that this bug is already closed. I spent > non-trivial amount of time debugging it this morning because I found no open > bug describing the crash (in my case reported against curl). Ok. I'll reopen bug 1311559, because that's the one with the package nvr in the subject.
(In reply to Kamil Dudka from comment #14) > Kai, could you please fill in the Fixed In Version field? I don't know which version we would use for that? The usual rules might not apply for this kind of scenario, because the bug isn't in a released package, and we aren't creating a fixed package to fix this bug. Rather, the fix was rel-eng tags.
I would expect a NVR that one should install to resolve the bug in question (being it upgrade or downgrade does not matter in my opinion).